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Chapter 1 



Introduction 

Since the launching of the Sputniks and the final awakening of the 
nation to the fact that Russia is producing approximately twice as many 
scientists and engineer;! as the U. S. whose technical training is apparently 
not inferior to our own (Golovin, I963), the importance of identifying 
creative scientific talent has received a great deal of national publicity. 
However, the need for creative scientific talent has been recognized by 
some psychologists and manpower specialists years earlier. As early as 
1955 , research had been accomplished by enough psychologists so that the 
first of the Utah Creativity Conference series could be held on the 
Identification of Creative Scientific Talent with National Science 
Foundation support, and since that time an increasing number of researchers 
have undertaken studies on creativity (Taylor, 1956, 1958, 1959s T-96l, 

1962, and 1963). The principal investigator of the present project and 
Robert Lacklen, personnel director of NASA, as early as 1956 had discussed 
that each had decided independently that biographical information would 
probably be the best single means of identifying creative scientific 
talent. Intermittent discussions were carried on until 1959 when the 
present project was initiated. Thus the present study had a long history 
before it was started. This report presents the results of a project 
supported by the National Aeronautics and Space Administration (NASA) which 
was designed to do something to improve the quality of scientific talent 
available to the space agency and to the nation as a whole— in some ways 
a small but nevertheless an important step toward the solution of a 
crucially important problem. 

When this study was initiated in 1959s biographical information was 
considered to be one of the most promising means of identifying creative 
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scientific talent. Previous research from a variety of investigators 
(D. Taylor, 1957? Hoe, 1951 9 1963? Sprecher, 1957 9 etc.) had indicated 
that biographical information was a potentially promising technique for 
the identification of creative scientific talent, although no one had 
made a definitive attempt to exploit this avenue of approach. The approach 
had, however, demonstrated its usefulness in a variety of other settings 
for predictive purposes, e.g., for identifying successful salesmen 
(Kornhauser and McMurry, 19^1), for predicting academic success (Fittner, 
19^5? Sorenson, 1950? Hansen, 1950; for identifying leadership ability 
in the Army after World War II ^Adjutant General °s Office, 19^6), and 
others. 

The intent in this present study was to exploit the biographical 
approach and thus to determine and more fully understand the experiences, 
backgrounds, opinions, self-images, attitudes, etc,, which would aid in 
differentiating between the highly productive and/or creative scientists 
and those who were less productive and/or creative. In essence, the 
study attempted to determine what types' of life history situations— 
self -perceptions, attitudes, etc.— tended to be more common to the more 
successful scientists and to ascertain the results of these situations 
as they were personified in the individual scientists. 

When the biographical characteristics, experiences, self -descriptions, 
etc., were identified which would differentiate the more successful 
scientists from those who were less successful, the practical goal was 
to utilize these characteristics in developing an easily administered 
and scored biographical inventory which would aid in the identification 
of scientific talent at the college level. Hopefully, the inventory could 
be rewritten for tho early high school level so that it could be used 
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as a vocational guidance instrument, and thereby, high school students 
who had scientific potential could be encouraged to further their 
development. 

Prior to the initiation of this large-scale NASA-supported research 
project, two studies were under way at the University of Utah lo explore 
the relationship of biographical information to scientific accomplishment. 
The first was completed by Ellison (i960, 1962) in which 527 biographical 
items wore administered to 71 advanced graduate students in the physical 
sciences. A large number of items was used to explore more adequately 
the potential of this type of instrument. Empirical keys wore constructed 
to predict ratings of creativity, productivity, and the general character- 
istics of a successful scientist. An item alternative analysis was 
carried out and the resulting initial validities were very high, ranging 
from .91 to .94. No cross validation was attempted in view of the small 
sample size. vBased on the results of this study a new form of the 
Biographical Inventory was constructed and administered as part of a very 
intensive study on 107 Air Force Scientists by C. W. Taylor, Smith, 
Ghiselin, and Ellison (196l). /^Empirical keys were developed via an item 
alternative analysis on the basis of the following six criteria} judged 
work output, supervisory ratings on overall performance, productivity in 
written work, supervisory rating of drive-resourcefulness, originality of 
written work, and status-seeking :j organiz at ion-man 55 tendencies. As before, 
the initial validities obtained were very high for. all six criteria and 
again cross validation was not attempted in view of the small sample 
size, but the best items from both this study and the one by Ellison 
were identified and retained for future use in the NASA project. In this 
study of Air Force scientists various a priori biographical keys proved 
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to be among the best predictor scores used in the study* These keys were 
developed on the basis of the study by Ellison and also on the basis of 
the current state of knowledge of creativity research at that time* 

In both of these studies, the items that were keyod and retained for 
use in future research were somewhat arbitrarily selected. In other words, 
they wero not identified strictly in terms of the usual statistical 
significance requirements. This was done with the conviction that a 
consistent relationship even in the low levels of validity across studies 
and samples was a better method of item selection in the long run than 
a statistically significant correlation in any one study. All items so 
identified across a series of studies would probably meet the requirements 
of statistical significance as the sample size increased. The typical 
correlations between the item alternative and the criterion on the items 
retained for future research tended to bo rather low, ranging from #20 to 
.40 with a rather small percentage (but at least 5$) of the sample choosing 
each alternative. The items so selected to form a longer combined test 
resulted in the high initial validities, even though each item alternative 
accounted for only a small percentage of the variance. We have sometimes 
described the Biographical Inventory with its many items and alternatives 
as an instrument consisting of a great many little oars, with each oar 
pulling only slightly in the right direction, but with all the oars in 
concert a powerful pull is exerted* The approach has some actuarial 
features in that experience tables have been constructed with information 
about each item so that the valid information is utilized to the fullest 

i 

possible extent* 
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Chapter 2 

The Form A Study of the Biographical Inventory 

Based primarily on the two prior studies, a new form (Form A) of 
the Biographical Inventory (hereafter called the Bl) was constructed for 
administration to NASA scientists. The instrument, as it was administered, 
consisted of 300 multiple choice items with the majority of the items 
containing either 4 or 5 alternatives. The three hundred items were 
subjectively classified into four sections. Additional information about 
the Inventory can be gained by examining Figure 1 which shows the sections 
of Form A of the BI, a brief description of the item content in each 
section and the number of items per section. By inspecting Figure 1, it 
becomes readily apparent that the items were very heterogeneous in nature. 
Thus the BI is not restricted to a narrow definition of biographical 
experiences. From our perspective the biographical approach should attempt 
to measure not only previous life history experiences (including past 
environmental effects on a person), but also assess the outcome or 
manifestation of the heredity-environment coronation as it is personified 
in the individuals studied. /Thus, almost any trait, life history situation, 
experience, or self -description, etc., that was thought to have some 
relevance to the problem was considered if it could be cast into multiple 
choice form. The four and five alternative multiple choice format was 
utilized for a number of reasons: (a) it was more adaptable to describe 

life history situations and experiences: (b) it was more acceptable to 
the scientists: (c) it permitted a more detailed analysis, which could 
be reduced to a lesser number of alternatives if the item analysis re- 
sults so indicated: (d.) it provided an opportunity to see if any non-linear 
relationships existed between the alternatives of an item and the criteria. 
If there v re any non-linear characteristics, they could be scored 



Figure I 

The Sections of the Biographical Information Inventory (Form A) 
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accordingly to maximize the relationship of the item to the criteria. 

This point will be dsicsused in greater detail in a later report. 

Description of the Criteria . In the selection and development of the 
criterion measures the intent was to use all available information at 
each research center visited and to collect a few well-constructed 
performance measures of our own for research purposes only. At the NASA 
research center where Form A was administered* an official rating score 
was available for each scientist included in the sample. This score 
was used by the research center to assess the overall performance of the 
men. The rating was made by the men 5 s immediate supervisor and was 
reviewed by a higher level supervisor. This was the only criterion score 
which was based on the evaluation of two supervisors. 

Since the dominant emphasis in this study has been the identification 
of creative scientific talent 9 a special effort was made to develop a 
criterion form to assess creative performance. Lacklen (1958) and Ghiselin 
( 1963 ) kad independently formulated similar definitions for measuring the 
creativity of a contribution 9 i.e . 9 in Ghiselin 5 s formulation ^the degree 
to which a contribution restructures our universe of understanding” was 
the measure of a creative contribution and in Lacklen 5 s formulation ”the 
extent of the area which each contribution underlies s :s was the standard of 
measure. These conceptualizations were utilized as the basic rationale in 
the construction initially of a Creativity Check List* and later of a 
seven- step Creativity Rating Form. The Creativity Check List was combined 
on the same sheet with a Productivity Check List. Both check lists were 
adapted from tho *rork of D. Taylor (1958) wb° developed the scales using 
the Thurstone technique for attitude measurement. In the case of the 
Creativity Check List 9 modifications were made in accordance with the work 
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of Lacklen and Ghiselin as mentioned previously. The criterion form 
consisted of a series of statements with each statement having a pre- 
determined numerical scoring value, However 9 these scoring values were 
unknown to the rater or supervisor. If the statement applied to or 
described the man being rated 9 it was checked by the rater 9 otherwise it 
was left blank. The man°s rating on each typo of performances i,e, s 
creativity and productivity 9 was the median scoring value of all of the 
statononts that were checked,^ 

The sum of each man°s publications and patents was also used as a 
criterion score. These two criterion scores were combined because at 
that time a computer program was not available which could handle more 
than three criteria at a time. This score was obtained from the scientists, 
hence, it must be interpreted with some caution since some distortion 
could have occurred, A recent study by Miltzer and Slatzer (1963) 
bears upon this point. In their study the correlation between scientists 0 
reports of the number of their publications and an actual count of the 
number of their publications was found to be , 51 * These authors used this 
correlation to justify the use of reports rather than counts. This 
moderate relationship is certainly less than what might be desired but it 
is high enough to indicate that reports can be used with reservations if 
counts are not available. As stated before, some caution should be observed 
in the interpretation of these self reports. 

The G, S, level of each scientist has also been used as a criterion 
measure with the hope that to some degree it reflects the achievement of 

^Copies of these and other criterion instruments used in these 
studies may be obtained as long as the supplies last by writing the authors. 
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the scientists but in the Form. A study, once again computer program 
limitations did not allow the G. S# level to be used for item analysis 
purposes* It, of course, has the disadvantage that the G* S* level is 
partially a function of experience and it may also reflect other qualities 
of the scientist which are not necessarily related to merit, as was reported 
by Taylor ot al, (1963)* 

The Creativity Rating Scale, which was designed to measure the same 
characteristics as the Creativity Check List, was administered three 
months after the Creativity Check List so that an estimate of the 
reliability of the two creativity measures could be obtained* On this 
form the rater checked the one descriptive statement which best described 
the man being rated. This form was filled out by the same raters as those 

who completed the Creativity Check List* 

The Administration of the Biographical Inventory * The administration 
of the BI was carried out smoothly and efficiently, thanks to the great 
cooperation cf all the scientists and other personnel at the research 
center. All of the higher level supervisor personnel of each research 
division participating in the study met with the psychologist from the 
University of Utah 2 who visited the research center and with other 
officials concerned* The nature of the study was discussed, togetner 
with the importance of obtaining accurate criterion ratings. The criterion 
forms and the Biographical Inventories were distributed at these meetings 
with instructions that they were to be returned in a weeks time. All of 
the ratings obtained were kept strictly confidential, i.e., they were 
used for research purposes only and no one at the research center had 

2 Robert Ellison was the field psychologist who visited the NASA 
research centers- Gary Cooley also participated in one of the field trips* 
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access to them. Code numbers were use' 1 on the Biographical Inventories 
to insure that the individual scientists would remain anonymous with only 
the visiting psychologist having access to the name for each code number. 

An information sheet was also distributed with each inventory explaining 
the nature of the study to the individual scientists. Out of the 455 
Biographical Inventories which were distributed, 357 were completed and 
returned, 11 were returned with incomplete information, and 49 of the 
scientists were on leave. Thus, there was a balance of only 34 scientists 
who did not respond within approximately a week to the request for their 
cooperation. This is an excellent example of the high degree of cooper- 
ation which has been characteristic throughout the entire research project. 

The educational background of tho scientists was as follows! 24$ had 
a bachelor 0 s degree, 3 8$ had some graduate training but r.o graduate degree, 
13$ had a master 0 s degree, l6$ has some graduate work beyond the master 
degree, and 9 p has Ph.D. degrees. The particular areas of scientific 
research and engineering under investigation at this research center 
dealt with problems of propulsion for both atmospheric and space flight. 
Also under investigation were chemical, electrical, and nuclear propulsion 
systems, as well as 'more exotic schemes such as those involving thermal 
and solar conversion processes. 

The Data jftnalyjsis . A double cross-validation study was carried out 
as follows! the total sample of 357 scientists was numbered in 
consecutive order. All of the odd-numbered scientists were arbitrarily 
assigned to Sample I and all of the even-numbered scientists were assigned 
to Sample II. The sample size was 179 for Sample I and 178 for Sample II. 

An item analysis was then carried out for each sample independently. 
Due to computer limitations at that time, only three criteria could be 
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used in the item analysis for each sample* For Sample I these were: 
the Offical Rating Criterion, the Creativity Check List, and the 
Publications-and- Patents Criterion, For Sample 11 the criteria used in 
the item analysis were: *tfie Official Rating Criterion, the Creativity 

Check List, and the Productivity Check List, In the item analysis, 
biserial correlations were utilized to determine the relationship of every 
alternative of each item to each of the three criteria. Since the 
Biographical Inventory had 300 questions and most of the questions contained 
five alternatives, there were approximately 1300 alternatives in the 
Biographical Inventory, The relationship of each of these alternatives 
to each of the three criteria in each sample was analyzed independently. 
Thus, approximately 9000 biserial correlations were computed. All of 
the computations were carried out on a Burroughs 205 medium speed computer. 
For sooring purposes an alternative was scored if at least 5$ 
the sample chose the alternative and if the correlation of the alternative 
with the criterion was .20 or greater, regardless of sign. This level 
was selected on the basis of convenience and tradition. In both of the 
studies which preceded this Form A study, this percentage value and 
correlation level were used with the highly promising results mentioned 
previously. In terms of statistical significance, this does not quite 
meet the requirements for the ,05 level of significance. The standard 
error for a biserial correlation where the correlation is assumed to be 
0, and 5$ of the sample respond to the alternative with a sample size of 
179 is ,118 so that a correlation of ,23 would be significant at the ,05 
level. If more than Sf° of the sample chose an alternative, then a lower 
correlation would be significant at this level. 
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If a correlation was plus ,20 or greater,, a weight of +1 was used, 
and a correlation of minus ,20 or greater was scored - 1 , In this manner 
separate positive and negative scoring keys were constructed for each 
criterion. The literature did not reveal any clear superiority for a 
more complicated weighting system (for example, see Guilford, 1950 > 
pp, 537-542). The sectional breakdown of the BI was also retained in 
the scoring keys 5 that is, for each criterion, tithe scores were as follows: 
Section 1— Developmental History up to Age 21, Section 2— Parents and 
Family Life, Section 3~ Academic Background, Section 4— Adult Life and 
Interest, and a Total Score which was the algebraic sum of these four 
section scores. The empirical keys that were developed on the basis of 
Sample I were then applied to Sample II and vice versa so that a double 
cross-validation design was carried out. In order to determine the 
magnitude of the initial validities, the Official Rating Key which was 
constructed on the basis of Sample I was also applied to Sample I, 

Seven a priori keys were also included in the data analysis. These 
keys were based on two studies mentioned previously (Ellison, 1960$ 

Taylor et al, , 1961) and upon the basis of clinical judgment. The best 
items from these two previous studies were given to two clinical 
consultants, together with definitions of seven traits thought to be 
important for creative performances in science. Their task was to classify 
each item into the one most appropriate category that they thought the 
item measured. All items on which there was no agreement between the 
two judges were not retained for this analysis. The seven traits were 
as follows: Professional Self-confidence, Low Sociability, High Self- 

sufficiency, Dedication to Work, Liking to Think, Intellectual Thoroughness, 
and Inner Directedness, These keys were applied to both Sample I and Sample II. 
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After the item analysis on both Sample I and Sample II was completed 
and the keys were constructed for each sample, two additional sets of 
keys were developed* These were a combined Official Rating Key (with 
subscores for Sections 1, 39 ^9 plus the Total Score) and a combined 
Creativity Check List Key (with subscores for Sections 1, 39 ^9 plus the 
Total Score)* These keys were composed of items which had an average 
correlation of *20 or greater, with similar signs across the two samples 
and to which at least 5$ of the total sample (Sample I plus Sample II) 
responded* Thus, these keys, when applied to both Sample I and Sample II, 
were indicative of the initial validity of the instrument if an item 
analysis had been carried out on the full sample of 3 57 scientists* 

Two additional variables were also included in the Sample I analysis* 
The first of these was the total number of times each scientist was 
scored on the Sample Two Official Rating Key regardless of sign* Although 
no definitive evidence is available, it is suspectod that this score 
reflects, at least to some degree, the independence of the subjects 
completing the Biographical Inventory, since the items with the low 
percentage values (i,e,, the percentage of the sample which chose each 
alternative) are the ones which tend to be scored on the inventory* Thus, 
those scientists who tended to mark the alternatives which were chosen 
less frequently by the other scientists, as a whole, would tend to 
be higher on this score* The .other additional variable included in the 
analysis was termed an Off-line Measure* This score was an exploratory 
attempt to develop a correction score for those scientists who tended 
to be classified in an incorrect manner by their biographical score; that 
is, the scientists could be classified in terms of those who either had 
a high biographical score in relation to their criterion score, those 
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who had similar biographical and criterion scores, and those who had a 

low biographical score in relation to their criterion score* As seen from 

another perspective, this score measured how much error was made on each 

scientist in predicting his criterion score from his biographical score* 

Briefly, the procedure used in developing this score was as follows* 

on Sample I the Official Rating Criterion Score and the Sample II Official 

Rating Key Total Score were both equated in terms of means and standard 

deviations* Then the Key Score was subtracted from the Official Rating 

Criterion Score and a large enough constant was added to make all scores 

positive. In this manner a new distribution was created which described 

the subjects according to the type and the degree of the error in the 

initial prediction. This score was constructed for later use in an item 

analysis and it was included in this analysis to ascertain its 

characteristics. The plan was that the key derived from it would then be 

cross validated on an independent sample with the hope that it would aid 

in identifying an additional portion of the criterion variance, i.e,, 

3 

the false positives and the false negatives. 

All the correlations between the variables for each sample were 

4 

computed on the IBM 7090 at Western Data Processing Center. The validities, 

^ The reader may recognize that this score bears some relationship to 
the standard error of estimates 

where: Y=Measured value of a case we 

are trying to predict 
Y fl =Predicted value for each case 
i (XsUl N=The total numb or of cases 

N predicted 

4ihe authors wish to acknowledge the facilities and the cooperation 
that was made available at Western Data Processing Center, University of 
California, Los Angelos. 
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intercorrelations , means, standard deviations, and number of items per 
biographical score for Sample I are shown in Table 1 and for Sample II 
in Table 2. 

Results of the Form A Study . The Sample I criterion intercorrelations 
are shown for variables 1 through 7 in Table 1. Of particular interest 
are the correlations of the Creativity Check List, variable 2, with the 
Creativity Rating, variable 6, in Sample I. It will be remembered from 
the previous discussion that the Creativity Rating Form was administered 
three months after the Creativity Check List, and that the forms were 
radically different in nature \ thus, this correlation can be taken as a 
conservative estimate of th'- reliability of the Creativity Criterion 
measures. This correlation was ,69* It was definitely lower in Sample II 
due to the fact that complete data for the Creativity Rating were not 
available at the time this analysis was carried out and mean scores Were 
used as an estimate where missing data were present. 

In Table 1, the Sample I Official Rating Key scores (Sections 1 
through 4 and the total) which were applied to Sample I criterion measures 
are variables 8 through 12, They give the initial validity for these 
keys on this sample for the Official Rating Criterion, They range from 
.53 for Section 2 to .79 for the total score against this criterion. 

Variables 13 through JO in Table 1 were constructed on the basis of 
Sample II and thus, the correlations of these scores with the criteria 
were all cross validity coefficients. From pilot studies it was 
determined that tho Parents and Family Life Section, Section 2 of the 
Biographical Inventory, consistently did not hold up on cross validation. 
Thus, in those scores a total score was computed with and without Section 
2 for each key. Table 1 shows that Section 2 of each of the three sots of 
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criterion keys consistently had either negative or essentially zero 
validities. Section 4, The Adult Life and Interests Section, consistently 
had the highest validity of any of the other sections, followed by 
Section 3 (the Academic Background Section), then Section 1 (the Develop- 
mental History Section), and finally Section 2 (the Parents and Family 
Life Section). The total score cross validities were .55 f° r the Official 
Rating Key score against the Official Rating Criterion? ,56 for the scores 
from the Creativity Check List Key against the Creativity Check List 
Criterion and .37 for the Productivity Key against the Productivity 
Criterion. 

The combined keys, that is, the keys that were based on both Sample I 
and Sample II, are variables 31 through 38 in Table 1. These are the 
keys that would be especially useful in further research, since they 
represent the best items from the analysis of both samples. These 
validities are somewhat higher than those obtained from the cross validated 
keys and, not as high as the Sample I keys applied to Sample I, as one 
would expect, since this is not the same as a regular cross validation. 

As mentioned previously this was an estimate of the initial validity of 
the instrument if an item analysis had been performed on the total sample 

of 357 scientists. 

The a priori personality keys were somewhat disappointing, as only 
four out of the seven keys correlated significantly at the .05 level with 
the criteria and two of these had only a low relationship though these 
were with the creativity criteria only. The Professional Self-confidence 
Key, which contained 17 items, proved to be the most valid a priori key 
with a validity coefficient of .47 with the Creativity Check List Criterion. 
The other a priori key which had substantial validity was the Intellectual 
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Thoroughness Key which correlated .41 wioh the Creativity Chock List 
Criterion. The other two keys which produced significant criterion 
correlations were the High Self-sufficiency Key and the Liking to Think 
Key, 

The Number of Times Scored on the Official Rating Key correlated 
•33 with the Official Rating Criterion and also correlated significantly 
with all of the other criteria, although not as high. This preliminary 
analysis of the Off-line Measure also proved intriguing since scores 
from this measure correlated ,48 with the Official Rating Criterion and 
-.46 with the total score of the Official Rating Keys thus, it was hoped 
that when an item analysis was carried out against this Off-line Measure 
the biographical key derived from such an analysis would correlate 
positively with the criteria and negatively with the key, and thereby 
contribute substantially in a multiple correlation equation. A thorough 
discussion of the results of these attempts to devolop correction scores 
will be presented in a later report. 

The results for Sample II are similar to Sample I although the 
cross validities in general were not quite as high. The cross validity 
sufficient for the total score of the Official Rating Key against the 
Official Rating Criterion was ,55. For the Creativity Check List Total 
Score Key the cross validity coefficient was ,48 against the Creativity 
Check List Criterion, and the Publication and Patent Keys total score 
correlated .35 with the Publications and Patent Criterion. 

The Official Rating Criterion was somewhat more predictable than 
the other criteria in the study, and the keys that were constructed on 
the basis of this criterion also gave the highest cross validities 
in the study. In Table 1 the correlation of variable 18, the Official 
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Rating Key Total Score without Section 2, with the combined Creativity 
Rating Criterion was ,59j whereas the Creativity Check List Total Score 
without Section 2 correlated .57 against the same criterion. The stability 
of the scores from the Official Rating Keys and the predictability of 
the Official Rating Criterion is probably due to the fact that two 
raters completed these ratings, whereas in the other ratings only one 
supervisor was involved. 

When one considers the magnitude of these cross validity coefficients 
that were obtained from only a single instrument, the Biographical Inventory, 
it is apparent that they represent a substantial degree of prediction. 

For comparative purposes , an illustration may be given. Psychologists 
have been working for some forty to fifty years on the prediction of 
grade-point average. A wide variety of tests, measuring a multitude of 
traits, both personality and intellectual, have been used in an attempt 
to predict this criterion. Yet, after some forty or fifty years of 
research, the typical validity coefficients that are reported in the 
literature for the prediction of grade-point average tend to range from 
.40 to .60. Usually, correlations of this magnitude are based on not 
just one instrument, but on two or three and perhaps several. Thus it 
can be seen that with just the Biographical Inventory a level of prediction 
against job performance criteria has been achieved that is very favorably 
compared with the results obtained from a problem that psychologists have 
been working upon for some forty or fifty years. Even more impressive 
evidence can be gained when the unreliability of the criterion is considered. 
Take, for example, the correlation of .69 between scores on the two 
creativity criteria mentioned previously that were obtained some three 
months apart. This correlation is probably a conservative estimate of 
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the reliability of these scales although Guilford (1956 a*p» 478) states 
that the reliability of ratings, even good ones, is about .60. But in 
any event, if the validity coefficient of a single biographical test 
score of .59 against the combined Creativity Criterion is corrected 
for unreliability in the criterion only, the cross-validity coefficient 
would increase to approximately .70. This is a very high level of 
prediction under any circumstances, especially for such an important external 
criterion as creativity. 

The a priori personality keys when applied to Sample II were not as 
valid as they were in Sample I, since only 2 of the variables had * 
significant correlations with the criteria. These were the Professional 
Self-confidence Key with a validity coefficient of .48 against the 
Creativity Check List Criterion and the Intellectual Thoroughness Scale 
with a validity coefficient of .39 against the Creativity Check List 
Criterion. Scores from these two keys, which were designed to predict 
creative performance, were able to predict all the criteria at the .01 
levels however, they had higher correlations with the creativity criteria 
than with the other criteria used in the study. 

Discussion of the Results in the Form A Study . The generally high 
intercorrelations among the criteria were in marked contrast to the study 
by Taylor et al. (1961) on Air Force scientists. In Taylor °s study, the 
criteria were obtained from a variety of sources yielding many different 
kinds of scores. In the present study, all of the criterion scores had 
their origin from either the scientist 0 s immediate supervisor or from 
the scientists themselves, so that the number of sources utilized was 
considerably less than in Taylor 0 s study. The Publication and Patents 
Criterion and the G, S. Level Criterion were, generally speaking, more 
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independent of the other criteria used in this study* The relative 
independence of these criteria from the other criteria of success in 
science is a finding which is similar to the results found by Taylor although 
they were more related to each other than in the study by Taylor ©t al* 

In Taylor 0 s study, multiple criterion measures were factor analyzed and 
a factor ©merged which was identified as Productivity in Written Work, 
which was relatively independent and separate from a Quality of ’Written 
Work, and also from supervisors 0 ratings of performance on various traits* 

In Taylor’s study, G* S* Level also emerged on a separate factor which 
was labeled Current Organizational Status. Again, this factor was 
independent from the majority of the other criteria used in the study, 
i.e., it contained no high loadings from the measures which defined the 
other factors of Quantity of Work, Originality of Written Work* etc. 

As one might expect, the magnitude of the initial validity coefficient 
is partially a function of the sample size. In looking across the college 
study by Ellison (i960), the /dr Force Study by Taylor et al* (1961), 
and the present study, there has been a consistent decline in the magnitude 
of the initial validities obtained as the sample sizes have increased. 

In the college study, with an N of 71 9 the highest initial validity was 
.94 and in the present study, the initial validity was .79 when the 
Sample I keys wore applied back to Sample I, (N = 179)* When the combined 
keys, that is, those that were built on both Sample I and Sample II, 
were applied to both subsamples, the average initial validity for the 
total score of the Combined Official Rating Key was ,66, This was with 
a sample size of 354, Therefore, it seems likely that, at least with 
the present inventory, the level might be reached where the initial 
validity would be only slightly greater than average cross validity 
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of the two sub samples, although the size of the sample where this would 
bo true is as yet unknown. It is recognized that other variables 
also have a bearing on this question, such as the number of items and 
alternatives in the questionnaire, etc. 

The disappointing performance of the Parents and Family Life Section 
of the BI is hard to explain. Certainly a variety of theoretical for- 
mulations as well as previous studies, partially those of Roe (1951 » 

1963), have indicated that the socialization procedures of the parents 
should have a substantial bearing on the outcome and development of 
the children. Indeed, other investigators have constructed instruments 
to measure these effects. Examples are the Parental Attitudes Research 
Inventory by Schaeffer and Bell (1958), and the questionnaire developed 
by Roe (I960), both of which were designed to specifically measure the 
effects of the parents and the family life upon the socialization of the 
children. In the present study there are a number of possible explanations 
for these results. Perhaps the present study failed to include appropriate 
items, or the items themselves were poorly constructed. Another reason 
that appears more probable is that when any one facet of parental behavior 
has been measured it does not provide enough information about how the 
other parental characteristics interact due to the complex network of 
interactions that exist between the subjects parents. Thus, by itself, 
the characteristic being measured appears unimportant. Perhaps the most 
likely explanation is that this area is extremely difficult to measure 
in any fashion if results are to be obtained that indicate that the 
socialization procedures have an infTtuence on the performance of 
scientists in a research setting. 



o 
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The relatively low cross validities obtained from scoros on the 
Productivity Check List Keys against the Productivity Criterion were an 
unexpected phenomena* Tho highest cross validity obtained with one of 
these keys against the Productivity Check List Criterion was .37 as shown 
in Table 1. This was the Total Score without Section 2. These results 
may have been due to the implicit selection procedures of the investigators, 
as they selected items that were primarily related to the creativity 
aspects of the criteria. An alternate possibility is that the Productivity 
Check List Criterion, for some reason, was less reliable than the other 
criteria and hence less predictable* This possibility has some support 
from other sources* For example, in Sample I the Productivity Check List 
Criterion correlates slightly higher with the Official Rating criterion { *68) 
than doos tho Creativity Check List Criterion ( *65)* Yet, when the Official 
Rating Key for Sample I was applied to Sample I, it correlated higher 
with the Creativity Check List than with the Productivity Check List, i*e*, 
the total score of the Official Rating Key, constructed on the basis of 
Sample I and applied to the Sample I criteria, correlated with the 
Creativity Check List Criterion .69 while the same key correlated .59 
with the Productivity Check List Criterion. Thus, from another source, 
there was evidence that the Productivity Check List Criterion was less 
reliable and hence more difficult to predict from the items included in 
the Biographical Inventory. 

As mentioned before, the validity coefficients of the & priori 
scores were rather disappointing. This was a preliminary attempt to 
determine the construct validity of some of the items in the inventory. 

What was really needed was a factor analysis of the best biographical 
items with landmark variables included as well as the criterion scores. 
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Such a study has recently been completed and the results will be presented 
in a later report, 

A rather intriguing finding that came out of the analysis of the 
a priori scores concerns the relative magnitude of the validity coefficients 
of these scores predicting the Official Rating Criterion and the Creativity 
Check List Criterion, There is a noticeable tendency, especially in 
Sample I, but which is also discernible in Sample II, for the Creativity 
Criterion to be more predictable by these a priori scores than was the 
Official Rating Criterion, Yet, as mentioned previously, the Official 
Rating Criterion was overall the most predictable criterion in the 
study in terms of the empirical keys which were constructed. This 
evidently illustrates again the implicit techniques of the investigators 
in constructing and selecting items which were somewhat more related to 
the Creativity Criterion than the other criteria used in the study. 



Chapter 3 

The Form B Study of the Biographical Inventory 

A new form of the Biographical Inventory was constructed (Form B5 
based on the best items of the previous administration. The ^deadwood 1 ’ 
items were eliminated from the previous form of the Biographical Inventory 
to make room for new items to be evaluated in this form with the hope that 
the percentage of 55 livewood :5 items in the new set of J00 items would be 
increased. The best itoms that were identified from Form A, that is, 
those items which worked consistently across both subsamples were subjectively 
reclassified into four substantive categories which appeared to be the 
most descriptive global categories of what the items were measuring. The 
criterion for the selection of an item into Form B required that each 
item have one or more alternatives with an average validity coefficient of 
.20 or greater with either the Creativity Criterion or the Official Eating 
Criterion across both of the subsamples with at least 5$ of the total 
sample responding to the valid alternative. Other carry -over .itoas 
included in Form B were those retained for control purposes such as the 
number of years of education, age, etc., and a few other items of inter- 
est from a theoretical perspective. The subscores into which the items 
were subjectively classified were as follows, (l) General Intellectuality, 

(2) Independence, (3) Professional Self-confidence, (4) Miscellaneous, and 
(5) the algebraic sum of the previous four scores. Two psychologists were 
responsible for this sorting of the items. In cases where there was 
disagreement as to the category in which the item should be placed, a 
discussion was carried out and, if disagreement still resulted, the item 
was classified into the miscellaneous category. These keys then represented 
the best accumulated information from the first study of the Biographical 




Inventory. 
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In the second study, a revised form of the Creativity Rating Scale 
vas administered. It was the only criterion score collected for research 
purposes only. Available at the research center were a number of official 
evaluations and data were collected on the most appropriate of these « The 
•elected ratings were on the following characteristics: Knowlodge of Work, 

Initiative, Judgment, Industry, Reliability, and Cooperation. In addition, 
information was also collected from the scientists on the number of 
publications and patents that each had produced. 

At the second NASA research center 370 Biographical 
Inventories were distributed and 35^ were returned, leaving only l6 scientists 
who were not accounted for. Again this was considered to be an extremely 
satisfactory return. Once more the cooperation and interest of every- 
one concerned was very high and greatly appreciated. The 35^ returned 
included nine who were on leave and four which were incompletely filled 
out. This left a maximum of 341 cases available for processing for 
which criterion data were available on our creativity rating. However, 
due to missing data on the criterion scores which were available at the 
research center, the final sample used in the data analysis was 300. Again 
the same procedure was followed in analyzing the data. Briefly, the total 
sample of 300 scientists was split in two subsamples of 148 and 152, as 
in the previous administration, and an item-alternative analysis was 
carried out for each sample in a double cross-validation design, 

For the processing of these data a new computer program wa$ utilized 
as well as a new computer. Arrangements were made at the Western Data 
Processing Center of the University of California at Los Angeles for the 
processing of the data. A biserial, point biserial item analysis program 
was written by Mr. Dan Morris , now of the University of California at 
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San Diego 9 specifically for analyzing the data. Since this program resulted 
in a substantial time saving, as well as providing additional research 
information, it will be described briefly. A maximum of 13 criteria may 
be used in the item analysis program. The computer first computes the mean 
and standard deviations of the criteria, then the relationship (either 
biserial or point biserial correlations or both) of each alternative of 
each question to each of the 13 criteria is analyzed individually. A 
standrad error for a bi serial correlation is also computed for each 
alternative. The absolute number of scientists choosing each alternative 
is printed out as well as the percentage of the sample choosing each 
alternative. The program also computes the mean and standard deviations 
of the distribution of the scientists across the alternatives of each 
question. After completing these computations, the program computes 

/ 

the intercorrelations among the criteria and then scores a second 
sample of scientists on the basis of the item analysis performed on the 
first sample. The scores of each scientist for each of the criteria 
are then printed out. Several options are available on the type of print- 
out desired. For example, a separate score was obtained for each of the 
four sections of the Biographical Inventory, namely, the Developmental 
History Section, the Parents and Family Life Section, the Academic Back- 
ground Section, and the Adult Life and Interests Section; finally a total 
score was computed based on the previous four r After these scores are 
computed, the cross validity coefficients are computed for each key against 
each of the criteria used in the analysis. After the program was 
!, debugged,” the computer performed all these operations, which included 
the calculation of approximately 36,000 biserial correlations, the means, 
and standard deviations of the questions and the criteria, scored both 
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samples of scientists according to the magnitude of the correlations , and 
computed the cross validity coefficients and the criterion correlations 
in approximately 12 minutes. The use of computers has, needless to say, 
greatly facilitated this research. In fact, without computers such research 
could only be approximated. The intensive analyses that have been carried 
out have added greatly to the results that have been obtained. 

The criterion intercorrelations, cross validities, and standard 
deviations for Amos Sample I are presented in Table 3. The first twelve 
variables were held up as criteria in an item analysis on the two samples } 
however, no results are shown from the item analysis of variable 12, the 
Form A Key total score. Variables 13 through 6? were the empirical keys 
which were constructed from an item analysis of Sample II and then applied 
to the scientists in Sample I. In other words, all of the correlations 
in this section are cross** validity coetxicionts. Variables 68 through 75 
were various a priori predictor keys derived as follows: variable number 

68, labeled Off-line Key #4, was one of the correction scores developed 
with the hope that it would help to identify those scientists with a large 
amount of error in thoir predicted criterion scores. Briefly, this score 
was obtained from an item analysis of Sample II where the criterion used 
was that portion of the criterion variance that had not been identified 
by the key from the first study. As mentioned previously, these correction 
scores will be discussed in a later report. Variable number 69, Rank in 
College, was an item which was taken from the Biographical Inventory and 
it read as follows: About what percentage of students did you surpass 

academically when you graduated from college? The alternatives were: 

A. 99$, B. 90$, C, 60$, D. 40$, E„ Don B t know. Variable 70 was a similar 
item but in this case the scientist described was high school porf or franco. . 
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Here the alternatives reads A* 99$, B. 90$, C. 80$, D. 60$ and E. 50 $ 
or less. General Intellectuality, variable 71, was one of the keys which 
was developed from the first study of the Biographical Inventory as 
mentioned previously. This also applies to variable 72, Independence, 
variable 73, Professional Self-confidence, and variable 7^, the Miscellaneous 
category of items. Variable 75, Independent Intellectuality, was strictly 
an a priori exploratory attempt to determine what relationship, if any, 
the learning of complex terms, symbols, and concepts on the students* 
own time would have to success in science. Examples of the terms to 
which the scientists responded whether they had learned them in school or 
on their own time were as follows: fugue, allegro, pollenation, volt, 

catalyst, spontaneous combustion, wolverine, electro magnet, AC and DC 
current, survival of the fittest, osmosis, and lepidoptera. If the 
scientists reported' that they learned these terms prior to twelve years 
of age outside of school they were given two points for each one; if they 
had learned these terms or concepts between twelve and eighteen years of 
age outside of school they were given 1 point for each term to which 
this classification applied. 

Results and Discussion of Sample I in the Form B Study . In Table 3, 
which presents the results from an analysis of Sample I, the first ten 
measures were all criterion scores and it will be noticed they were all 
characterized by moderately high interrelationships. The only exception 
was variable 3, Number of Patents, which was relatively independent of the 

other criteria used in the study. This variable, with a mean of .22 and 
a standard deviation of .58 was so skewed, with the overwhelming majority 

of the scientists having zero patents, that the relationships with other 

variables were to be expected. Scores from the Off-line Measure, variable 
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CRITHUON INTOCCRRELATIONS , VALIDITIES, MEANS, AND STANDARD DEVIATIONS 
FOR FORM 6 SAMPIX I (N = 148 N.A.S.A. SCIENTISTS)* 
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11, correlated .51 with the Creativity Hating and -.50 with the Creativity 
Key based on the first study which was variable #12, i.e., variable 12 
was the total score derived from Form A at the first administration. It 
is the sum of variables 71, 72, 73, and 74. The validity coefficient of 
.48 for this Form A total score against the Creativity Rating Criterion, 
was considered to be very satisfactory for a number of reasons. The key 
was built on scientists at a different NASA research installation with 
different research specialities and in an entirely different part of the 
country. In addition, the Creativity Rating was the criterion used in 
the second study, yet the biographical scores based on the first study 
were developed and validated on the basis of the Creativity Check List. 
Furthermore, it will be remembered that the Creativity Check List and 
the Creativity Rating (the latter was administered 3 months later) correlated 
.69 on the one sample for the first study upon which complete data wore 
available. Thus, in view, of the unreliability which was present in the 
criterion data, obtaining a cross-validity coefficient of .48 at a new 
research center was a gratifying research finding. This, of course, 
implies that the Biographical Inventory 0 s keys can be applied in various 
NASA research centers and that keys will not have to be built separately 
for each research installation. This same key based on the first 
administration also predicted most of the other criteria at the new 
research center with a satisfactory degree of validity. That is, the 
Form A key scores correlated .40 with the Number of Publications, .28 
with the Numher of Patents., *37 With G. S. Level, .38 with Knowledge 
of Work, .33 with Initiative, and .32 with Judgment. Although the 
correlations with the other criteria, Industry, Reliability, and Cooperation, 
were unsatisfactory, as will be discussed later, this was probably due 
to the inherent unreliability in these latter criterion measures. 
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Variables 13 through 67 are all empirical keys based upon Sample II 
and thus these are all cross- validity coefficients* The first four of 
these, variables 13 through 16, in Table 3 correspond to the sections of 
the Biographical Inventory as previously discussed, that is, the 
Developmental History Section, the Parents and Family Life Section, the 
Academic Background Section, and the Adult Life and Interests Section* 

In terms of how well they predicted the criteria, these sections are listed 
in terms of reverse rank order* This is somewhat in contrast to the results 
obtained on Form A, where Section One, the Developmental History Section 9 
was better than Section Two, the Parents and Family Life Section. This 
is probably due to the fact that in Form B Section Two was shrunk con- 
siderably so that the more valid items tended to remain while Section One 
was expanded with new and less stable items* The small difference could, 
however, be due to chance. The Total Score, variable 17, which was the 
sum of the previous four, had a correlation of .49 with the Creativity 
Criterion. This is in contrast to the cross-validity coefficients which 
were obtained in the first study, since there, the cross validities 
ranged in the mid fifties for the Creativity Key. On the other hand, 
it will be noticed that this Total Score correlates only .01 higher with 
the Creativity Criterion than the Form A Key that was built on the 
scientists at the first research center which, as previously mentioned, had 
a validity coefficient of .48 against the Creativity Criterion. This 
indicates a surprisingly high degree of stability in the scoring keys that 
were developed at the first administration since it was almost as satis- 
factory as the key actually constructed on the basis of the responses of 
the scientists at the second research center. These results certainly 
indicate a substantial degree of stability in the scores from biographical 
keys as the cross-validity coefficients obtained are remarkably consistent. 
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The cross validity of the Publications Total Score, variable 22, 
against the Publications Criterion was .55. It will be noticed that the 
Section Four Key actually had a correlation of .56 against the Publication 
Criterion, slightly higher than the total score. This occasionally happens, 
due to the fact that other sections of the inventory are not as predictive 
(i.e., not as discriminatory, some may have either zero or negative 
validity) and the simple additive sum is not the optimum way of combining 
the subscores. This issue will be finally resolved and capitalized upon 
when factor scores are obtained from a factor analysis of the biographical 
items in the inventory. The same phenomenon is evident in the correlations 
of Patents Key Total Score against the Patents Criterion which was .22 
and tho Section Four score which had a correlation of .30 against the 
Patents Criterion. Without taking time to discuss the rest of the cross- 
validity coefficients in detail, a brief summary follows? for the G. S. 

Level Key total score predicting the G. S. Level Criterion *56; Knowledge 
of Work Key total score against that criterion .35? the Initiative Key 
total score predicting the Initiative Criterion .28; the Judgment Key 
total score against the Judgment Criterion .37$ the Industry Key total 
score predicting the Industry Criterion .11$ the Reliability Key predicting 
the Reliability Criterion .07; the Cooperation Key predicting the Cooperation 
Criterion -.01 and the Off-line Key predicting the, Off-line Criterion .50. 

The comparatively lower cross- validity coefficients of the latter criteria 
which were collected at the second research center (Industry, Reliability, 
and Cooperation) are probably due to the manner in which these criteria 
were measured and the fact that the biographical items were primarily 
constructed and selected to predict other types of criteria. All of these 
traits were on a- single form and an inspection of the ratings after they 
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had been filled out revealed that there was comparatively little consistency 
in those ratings. Since the Biographical Inventory does predict the other 
criteria with comparatively higher cross-validity coefficients it appears 9 
then, that these lower validities for the latter criteria are primarily 
a function of the criteria themselves. A more impressive argument, and 
an intriguing finding also, about the keys which were constructed to predict 
those latter criteria. Industry, Reliability, and Cooperation, is that 
although they did not have any appreciable cross validities against the 
criteria they were constructed to predict, they did have at least moderate 
cross validities against the more reliable criteria. For example, in 
Table 3 the Reliability Biographical Key, constructed on the basis of 
Sample II and applied to Sample I has a cross-validity coefficient 
against the Reliability Criterion of only ,07, Yet the scores from this 
key correlated with the G. S, Level Criterion .53. It also correlated 
,48 with the Number of Publications, and ,38 with the Creativity Rating, 

The same tendency is also evident in Sample I which adds up to an impressive 
argument that the Reliability Criterion Rating is unreliable. Stated in 
other words it appears that the key built upon this criterion has more 
length and thus more reliability than the criterion itself. The same 
tendency is also true for the keys built to predict the Industry and 
the Cooperation criteria. 

The high cross-validity coefficients obtainod from the G, S, Level 
Total Score Key against the G. S, Level Criterion, ,64, the Creativity 
Criterion, ,51 9 and the Publications Criterion, .57* argue that even 
higher cross validities might be obtained if more reliable criteria 
were available. That is, the scores from the key constructed to predict 
the G, S, Level Criterion predicted the Creativity Criterion and the 
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Publications Criterion higher than the keys which were specifically 
constructed to predict these criteria— these results are cross validities, 
not initial validities* 

Variable 68, one of the correction scores and the first of the a priori 
predictors, correlated *21 with the Creativity Rating and -*07 with the 
Creativity Key* As mentioned previously these scores will be presented 
in detail in a later report but, for the moment, it will be noticed that 
this pattern of correlations would result in additional variance accounted 
for when added to the Form A Biographical Total Score Key* Variables 69 
and 70, Rank in College and Rank in High School, respectively, generally 
had very low correlations with the criteria, and they correlated slightly 
higher with the Creativity Key Scores from the first study* These results 
do not argue that academic achievement is a very good indicator or predictor 
of success in science* These variables of course 0 must be interpreted 
with caution since they are self-reports and may not be accurate descriptions 
of the scientists 9 actual rank in college and high school. Variables 71 
through 7^a inclusive, were the a prion, keys derived on the basis of the 
analysis of the scientists at the first study, as mentioned previously* 

Since they were designed and constructed to predict creativity it is no 
surprise that they do tend to predict creativity higher than the other 
criteria with Professional Self-confidence, Independence, General 
Intellectuality and the Miscellaneous Items Key being in rank order in 
terms of their predictive powers. Independent Intellectuality, variable 
75 9 was also explained previously* It is sufficient to note that it had 
insignificant correlations with all of the criteria except for the Off- 
line Measure with which it correlated -*23, evidently indicating that this 
score might make a small contribution to the identification of those 

/ 
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scientists that would not be accurately identified by their regular 
biographical scores. 

Analysis of Sample II in the Form B Stud?/ . The criterion intercorrelations, 
validities, and standard deviations for Sample II are presented in Table 
4. The variables in this table tend to parallel those discussed in Table 
3, except that certain variables were eliminated from consideration in 
order to make more room for other predictor scores. Thus, it will be 
noticed that certain section scores from the various empirical keys that 
did not have any significant validity coefficients across the criteria 
were eliminated. These results were obtained through pilot studies, and 
hence for the more thorough analyses presented here they could be 
eliminated to make room for other predictor variables. The a priori 
keys included in the analysis of Sample II are described below. 

Variable 63 , Extent of Participation, was based on the rationale that 
extensive participation in a variety of science-like activities prior to 
the age of 18, such as nature study, radio, electronics, photographic 
processing, etc., might be related to later achievement in science. 

Accordingly, 17 items wore selected from the Biographical Inventory which 
measured the frequency and extent of participation in these activities 
and were scored so that high participation would earn a high score. 

Academic Science Performance, variable 64, was a self-report on the 
level of achievement obtained by each subject during high school and 
college in five subject matter areas, namely, biological sciences, 
physical sciences, mathematics, engineering, and social sciences. 

Variable 65, Minimum Satisfaction, and 66, Dedication to Work, were 
both £ priori motivation scores. The former measured the degree of 
achievement necessary to obtain minimum personal satisfaction in three 



facets of science, namely, number of publications, level of original work 
and level of theoretical contributions. The latter & priori motivation 
measure, Dedication to Work, was also made up of three items s the extent 
to which hard work was felt to be the basic factor of success, the degree 
of absorption in work, and the extent to which work activities influenced, 
or interfered, with other aspects of daily living* 

Variable 67, Childhood Unhappiness, was of interest from a theoretical 
point of view, since the scientist is often characterized as one who has 
been reared in a rather cold emotional environment and/or his perception 
of his childhood is often described as being somewhat less than happy. 
Accordingly, this score consisted of seven items, all of which were scored 
in a continuous fashion to measure such characteristics of family life 
as the frequency of disagreements with the parents, the marital happiness 
of the parents, the frequency of emotional upsets during childhood, the 
satisfaction derived from the family life, etc. 

Variables 68, 69 , and 70, were the a gglori keys derived from the first 
study of the Bi .'graphical Inventory: General Intellectuality, Independence, 

and Professional Self-confidence, as discussed previously, 

Protestant Ethic Orientation, variable 71? is largely self-explanatory 
in that items were selected from the Inventoiy which were thought to be 
related to this conceptualized pattern of behavior. This score was made 
up of 9 items measuring such characteristics as achievement orientation of 
childhood friends, self-description of childhood ambition, the degree 
to which the parents encouraged thriftiness and saving, the importance 
of personal and economic success, etc. 

Variable 72, The Total Times Scored, was simply the algebraic sum 
of the number of times each scientist was scored either positive or 
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27 


09 


03 


21 


23 


19 


15 


03 


00 


-13 


-07 


32 


10.88 


1.1 


67. Childhood Unhappiness 


14 


11 


09 


05 


04 


09 


-01 


-01 


03 


-04 


-10 


22 


16.24 


3.* 


68. General Intellectuality- 


19 


32 


15 


20 


18 


19 


15 


06 


08 


01 


-59 


79 


51.26 


4. 4 


69* Independence 




40 


30 


15 


33 


51 


31 


24 


19 


10 


02 


-30 


6ffi 


49.39 


3. 


70. Professional Self-confidence 


50 


42 


19 


55 


42 


38 


34 


20 


20 


32 


-42 


90 


49.16 




71. Protestant Ethic Orientation 


-07 


18 


02 


06 


-51 


-02 


01 


09 


06 


06 


-34 


27 


27.51 


4*i 


72. Total Times Scored 


03 


04 


-09 


-18 


00 


07 


05 


-01 


03 


-03 


-10 


13 


45.05 


5.1 


73a Aee 




35 


44 


15 


59 


23 


13 


19 


16 


23 


22 


11 


22 


26.05 


•* 


74. Father's Education Level 


09 


05 


00 


-05 


08 


-09 


-10 


-10 


-07 


-07 


05 


03 


46,84 


21i- 


75. Subject's Education Level 


38 


29 


-07 


30 


32 


12 


12 


17 


08 


-02 


•01 


46 


23.75 


Ui 



* Decimal points onlttad, r <05 » .16, r <01 « .21 
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negative on the Sample I, Form B Creativity Key# This variable was the 
same as in the Form A study of the Biographical Inventory# Variables 73 1 
74, and 75 are largely self-explanatory, measuring the age of the subjects, 
and the amount of schooling completed by the father and by the subject# 

Tho results from the analysis of Sample II are presented in Table 4# 
Generali:; speaking the intercorrelations and validity coefficients for 
Sample II tend to follow closely the results already discussed for Sample I# 
Since much of the information in this table is similar to the results of 
the table previously discussed, they will be treated in a summary fashion 
pointing out only those of special interest# 

The Total Score on the Biographical Inventory from the first study, 
variable 12, correlated with the Creativity Criterion Hating #46# It 
will be remembered in the previous sample of the Form B study that the 
correlation was #48, thus, the average cross validity on the two samples 
was #47# This is additional evidence of the very stable nature of the 
Biographical key scores# The cross- validity coefficients of each of the 
empirically keyed scores against their appropriate criterion was as follows I 
the Creativity Key total score against the Creativity Rating Criterion 
.43, the Publications Key total score against the Publications Criterion 
# 5§., the Patents Key total score against the Patents Criterion #09, 
the G. S# Level Key total score against the G# S# Level Criterion .64, 
Knowledge of Work Key total score against the Knowledge of Work Criterion 
#38, the Irdtiativo Key against the Initiative Criterion #31, Judgment 
Key total score against the Judgment Criterion #30, the Industry Key 
total score against the Industry Criterion #06, the Reliability Key 
total score against the Reliability Criterion #13* the Cooperation Key 
total score against the Cooperation Criterion #06, and tho Off-line Key 
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score against the Off-line Criterion .54. Comparing these cross- validity 
coefficients to those of the Form A Total Score, variable 11, it will be 
noticed that the Form A Total Score from the first study did almost as 
well in predicting the criteria on the sample of scientists in the second 
study as did the keyed scores which were specifically constructed to 
predict the criteria at the second research center. In fact scores from 
the Form A Key correlated higher with the Creativity Rating Criterion 
(.46) than did scores from the Form B Creativity Key (.43). This is 
again evidence that the keys built on the samples of scientists in the 
first study do have a high degree of stability and can be used at more 
than one NASA research center. 

Variable 63 , the Extent of Participation Score, did not work as 
anticipated, since it correlated .05 with the Creativity Rating Criterion 
and also had essentially zero correlations with the rest of the 
criteria. There are a number of ways in which these results can be 
interpreted. To the extent that this score accurately reflects the 
extent of participation in a variety of science-like activities, then 
the results do not speak very well for Science Fairs and other activities 
which encourage an early participation in science. That is, the present 
results do not indicate that such participation is of a marked benefit. 

Of course neither do they indicate that such participation is in any way 
detrimental to the development of later competence in science. In any 
case it is an interesting question, one which should be followed up with 
additional research on Science Fairs and the people who participate in 
them. This Extent of Participation Score did correlate moderately high 
with the Total Biographical Score, .35 9 and with the Off-line Measure, 
-.40. This could indicate that the group of items selected for this 



measure were subject to a response set on the part of some of the scientists 
filling out the Biographical Inventory, That is, there was evidently 
a tendency for those who scored high on the Biographical Inventory to 
perceive these items either consciously or unconsciously as being 
indicative of the characteristics which should describe good scientists. 
Speaking in more traditional psychometric language, those scientists who 
scored high on this score would tend to be identified as false positives 
since thoir biographical score was higher than their criterion score. 

In terms of the correction score which will be discussed in a later 
report they could be thought of as exaggerators. Another way of viewing 
this score and the results that were obtained is that the scientist 
filling out the Biographical Inventory perceived these items the same 
way as the investigators did in designing this score. That is f they 
evidently felt these items would be characteristic of how successful 
scientists would respond or should respond to the Biographical Inventory# 
Academic Science Performance, variable 64, which was a self-report 
on the level of achievement by each subject during high school and 
college in five areas, biological science, physical science, mathematics, 
engineering, and social sciences, had essentially zero relationships 
with all of the criteria. But once again a moderately high correlation 
was obtained between this variable and the Biographical Score as well as 
the Off-line Criterion measure. To the degree that these self-reports 
are accurate they indicate, then, that achievement in high school and 
college in those areas had essentially a zero relationship to the 
criteria of success in science used in this study, but that obtaining 
A*s or high grades in these area v s was related to the score that was 
obtained on the Biographical Inventory, Perhaps this is a facet of how 



they perceive themselves, that is, the scientists who were more successful 
during their academic training also perceived themselves as being more 
successful as they filled out the Biographical Inventory. In other 
words they have a high self-concept, higher than what the criterion 
measures would indicate as realistic. These findings again raise questions 
and doubts about the appropriateness of our educational system. There 
have been enough studies which have conclusively demonstrated that, 
at best, the relationship between grade-point average and performance 
on the job is low, if statistically significant. These are not new 
findings, then, but they do add substance to the claim that our 
educational system needs to be improved, if grades are to be really 
representative of the type of performances that are needed in the 
world of work. (Taylor, 1958, 19635 Business Weok , 1962; D. Taylor, 1958). 

Variable 65, Minimum Satisfaction, and variable 66, Dedication to 
Work, were both short ja priori motivation measures * The Minimum Satisfaction 
Scale had rather consistent correlations across the criteria, correlating 
.25 with the Creativity Rating and .32 with the Number of Publications. 

The Dedication to Work Scale, while not having validities quite as high 
as the Minimum Satisfaction Score, was less related to the total score 
of the Biographical Inventory, correlating only .32 in contrast to the 
.56 correlation between the Minimum Satisfaction Scale and the Biographical 
Total Score. It is of interest to note that this Dedication to Work score 
correlated highest with Creativity, next highest with Dedication to 
Work, and then the G. S. Level Criterion. It had essentially zero 
relationships with the Number of Publications and Patents. This suggests 
that this score measures a task orientation and actual ^involvement in 

i 

work which docs not find expression in written reports. It seems to 
involve work for its own sake. 
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Varialbe 67, Childhood Unhappiness, did not have any significant 
correlations with the criteria although the correlation with the 
Creativity Rating was close to significant. It did correlate significantly 
with the Biographical Score, .22. Perhaps if this scale were lengthened 
the validity might increase to the point where it could be demonstrated 
that these items do at least have a theoretical significance if not a 
very practical significance for identification of scientific talent and 
other criteria of success in science. 

Variables 68 , 69, and 70, the three keys based on the first study 
of the Biographical Inventory, tended to have their usual pattern of 
significant validity coefficients with Professional Self-confidence 
being the most valid, followed by Independence and then General 
Intellectuality. The Professional Self-confidence Key correlated higher 
vith the Creativity Criterion (.50) than did the Form B Creativity Key 
Total Score (.43). I** terms of variance accounted for by each of these 

keys, there is a 7^ difference, with the key from a different research 
center being superior to the key that was developed and cross validated 
to maximize the prediction of the Creativity Criterion at the second 
research center. It is of interest to note that the General 
Intellectuality Score correlates higher with the Number of Publications 
than it does with any other criterion. It also correlates higher with the 
Off-line Criterion Measure, -.59, than do the other two a priori keys. 
Variable 71 was the Protestant Ethic Orientation, an a priori key. 
With the exception of a significant correlation with the Number of 
Publications, scores from this key did not have any other significant 
validities, except with the Biographical Key scores and the Off-line 
Measure. Those results might be interpreted in two ways. First, the 
key may actually measure a Protestant Ethic Orientation since the 
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significant correlations with the Number of Publications would bo positive 
evidence that one has achieved* or better yet, demonstrated that he is 
among the divinely chosen. If this rationale is realistic then the low 
negative correlation with Creativity is of interest since this would be 
consistent with the rationale that it is the demonstrable products or 
achievements that are important, and that an above-average level of 
creativity is not as important as positive demonstrable products of one’s 
effort 3 in fact, for some people such an orientation might actually 
detract from an orientation toward creativity. Second, this key could 
also bo interpreted as being similar to the Extent of Participation Scolre 
discussed above. That is, it could be measuring items that are primarily 
associated with a response set and as such this would explain the 
Correlation of .27 with the Creativity Key and the -.07 correlation with 
the Creativity Rating. 

Variable 72, the Total Times Scored on the Sample I Form B Creativity 
Key, did not produce the same results as an analogous score derived in 
the Form A Study? in fact this type of score in the Form A study 
correlated positively with most of the criteria, whereas in the Form B 
study it had essentially zero correlations, not only with all of the 
criteria, but also with the Form A Total Score (Variable 12) and with 
the Off-line Measure. No satisfactory explanation has yet been suggested 
for these results— apparently the results are inconsistent for this score 
as its keys is developed on each new sample. It may reflect to a small 
extent the ’’research climate” at various NASA research centers. This 
topic will be treated more thoroughly in a discussion of variable 75* 

Variable 73s Age, was included to determine its relationship to 
the criteria and to the biographical scores* Generally speaking, age 








correlated moderately high with most of the criteria but only ,22 with 
the Form A Key Score, Thus, scores on the Form A biographical key are 
not largely predicting age even though age is related to the criterion 
measures. 

Variable 74, Father’s Education Level, did not have any significant 
correlations with any of the other variables, thus indicating that for 
the scientists in this sample, the education of the father had no 
influence on the achievement of the son. This does not tell us anything, 
of course, about any effects of the education of the father on the son’s 
attending or graduating from college or choosing science as a profession. 

Variable 75, Subject’s Educational Level, had a pattern of surprisingly 
high correlations across the criteria and with the Form A biographical 
score. Those results are in contrast to those of Taylor et al, (1961), 

In this earlier study, they found that years of education had either low 
or zero correlations with the multiple criteria of success in science 
studied of which thero were 14 essentially independent dimensions. In 
view of these apparently contradictory results, additional evidence was 
gathered together to try to clarify this issue. Additional information 
was available from the item analysis results of the Biographical Inventory, 
In the Form A study, the level of education also had some significant 
correlations with the criteria but generally they were not as high as 
in the Form B study. In the Form A study, obtaining a B.A. or B 5 S. 
degree or loss had a low but negative relationship to the criteria while 
completing some graduate work but no graduate degree, or completing an 
M.A., or M.S, degree, or completing some graduate work beyond the master’s 
degree all had essentially zero relationships with the criteria. Completing 
tho Ph,D, degree, however, was positively related to the criteria, even 
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though only nine per cent of the sample in the Form A study had 
completed the Ph.D, degree. 

Another source of information on this topic comes from a recently 
I completed study by Taylor, Cooley, and Nielsen (19&3)* this study, 
which will be more thoroughly discussed in a later section of this 
report, high school and college students who participated in NSF-supported 
summer science training programs were given a revised form of the 
Biographical Inventory and other selected predictor instruments in an 

■i, 

attempt to "carry back” and validate instruments developed on mature 
| scientists to these younger age groups. In this high school study both 
age and grade level tended to correlate either zero or negative with the 
15 criteria of achievement in science that were used. Yet with age and 
grade level essonli^-Iy ”parti ailed out,” the Biographical Inventory 
with keys built on NASA scientists proved to be one of the bost predictors 
| used in the study. Thus, it appears that although the Form A Biographical 
\ Inventory Key is moderately related to level of education on this sample, 
this relationship appears to be at least partially contingent upon the 

v' 

I sample studied. And it also appears evident that the key contains 

much additional variance which is related to criteria of science achievement 

V" 

K 

other than that which would be identified by level of education. It is 

’fi 

also interesting to note that the keys constructed on the scientists in 
the Form B study have in recent research turned out to be "academic” keys. 
That is, in a study by Bunderson, Rigby, and Taylor, ( 1963)9 the Form 
[": B keys turned out to be the best predictor of combined academic and research 

performance in graduate school. Also, in the NSF study mentioned 
[ previously the Form A keys worked better in the research participation— 

type programs, while the Form B keys worked better in the classroom type 
of program. Those findings suggest that there may bo interesting "climate” 
differences between the two research centers studied. 
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THE FORM C STUDY OF THE BIOGRAPHICAL INVENTORY 



Another 300 item form of the Inventory was constructed in which the 
best items from the previous studies were used and some new items were 
added. This form (Form C) was administered to 76 9 scientists at a third 
NASA research center where ninety- seven per cent of the Biographical 
Inventories distributed were either completed or otherwise accounted for, 
still another excellent example of the cooperation which has been so 
characteristic of this study. The scientists at this third center conduct 
research in aerodynamics of re-entry vehicles, structures and materials 
for space vehicles, aircraft aerodynamics, fundamental plasma physics, 
and a wide variety of other areas. 

In contrast to the other two rosearch centers visited, there was no 
existing rating procedure at this center for the evaluation of scientific 
personnel. Promotions were handled by means of letters of recommendation 
and by meetings of those concerned. Thus, the criterion measures collected 
at this center may have been influenced by this comparative lack of rating 
experience. In addition to collecting information on the Number of Patents, 
the Number of Publications, and the GS Level attained, other criterion 
measures were collected from supervisors. These instruments were constructed 
by the investigators to evaluate the following traits i Quantity of Work, 
Skill in Getting Along with People, Creativity, and an Overall Evaluation. 
These criterion rating forms resembled those constructed for research 
purposes at the two previous centers. A sample of these forms is presented 
in the Appendix. 

The procedures followed at this third NASA center were again the same 
as in the previous studies. Criteria were collected from supervisors, the 
biographical inventory was administered to all the scientists, and after the 
total sample was divided into two sub-samples of 390 and 379 > an item 
alternative analysis was carried out in a double cross validation design. 
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Since all of the working items from the previous forms were retained 
in Form C, the scoring keys from the previous studies were also included in 
the analysis 9 that is# the General Intellectuality Key 9 the Independence 
Key, the Professional Self-confidence Key, and the Form A Total Score from 
the Form A Study, together with the Form B Total Score Key. The latter 
included all of the items to which at least 5 per cent of the total Form B 
sample responded and which also had an average correlation of plus or minus 
twenty or greater across the two sub-samples in the Form B Study with the 
Creativity Criterion. 

Results for Main Samples . Criterion means, standard deviations, 
intercorrelations, and cross validity coefficients for the keys from the 
previous studies together with the various new empirical keys constructed 
at this research center are shown in Table 5 for Sample One. Table 6 
shows the corresponding results for Sample Two. The Criterion inter- 
correlations are similar to those in the previous studies, with two 
exceptions. In the Form B Study where Nuriber of Publications was also 
retained as a separate criterion, its correlation with the Creativity Rating 
in one sample was .46 and in the other sample .38. However, in both of 
the samples in the Form C Study the corresponding correlations were 
considerably lower, .17 in Sample One and .21 in Sample Two with the 
Creativity Criterion. In addition, the G. S. Level Criterion correlated 
considerably lower with the Creativity Criterion in the present study than 
previously. For example, in the Form B Study, G. S. Level correlated .61 
in one sample and .55 in the other sample with the Creativity Criterion, 
while in the Form C Study the correlations of G. S. level with Creativity 

were .32 in Sample One and ,,36 in Sample Two. 

These results evidently suggest that the supervisors in their ratings 
did not take into consideration, to the extent that was done in the previous 
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Tabic 5 

CRITERION MEANS, STANDARD DEVIATIONS, INTERCORRELATIONS 
AND CROSS VALIDITIES FOR SAMPLE ONE IN THE 
FORM C STUDY OF THE BIOGRAPHICAL INVENTORY* 

(N = 390) 





No. 




1 


2 


3 


4 


5 


6 


7 

i 


Means 


s. : 




SAMPLE ONE CRITERIA 




















1 


Quantity of Work 
















9.21 


2.5 


2 


Skill with People 


65 


— 












8*92 


2.3 


3 


Creativity of Work 


69 


58 


— 










8.35 


2.6 
A 1. 


4 


Overall Work Performance 


85 


72 


83 


— 








9.24 


2.4 


5 


Patents 


08 


01 


13 


08 


— 






.23 


•8 


6 


Publications 


19 


21 


17 


2 C 


11 


— 




6.60 


7.3 


7 


G. S. Level 


27 


31 


32 


33 


13 


59 




11.91 


2.2 




SAMPLE TWO KEYS (Form C Study) 




















8 


Quantity of Work Total Score 


33 


31 


41 


38 


16 


28 


31 






9 


Skill with People Total Score 


32 


36 


32 


34 


05 


29 


38 






10 


Creativity of Work Total Score 


27 


24 


41 


34 


18 


21 


23 






11 


Overall Work Perf • Total Score 


31 


29 


4l 


37 


17 


27 


28 






12 


Patents Total Score 


20 


17 


31 


26 


17 


28 


33 






13 


Publications Total Score 


24 


24 


33 


29 


22 


59 


66 






14 


G. S. Level Total Score 


21 


24 


28 


24 


19 


57 


69 








OTHER PREDICTORS 




















15 


General Intellectuality 


13 


11 


17 


17 


17 


13 


14 






16 


Independence 


22 


15 


28 


22 


16 


13 


16 






17 


Professional Self-confidence 


28 


27 


41 


35 


17 


30 


29 






18 


Miscellaneous 


13 


09 


10 


14 


-01 


23 


24 






19 


Form A Total Score 


28 


24 


38 


33 


21 


26 


27 






20 


Form B Total Score 


14 


15 


26 


18 


11 


18 


20 








* Decimal points omitted, r 
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.13. 
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Table 6 

t 

CRITERION MEANS, STANDARD DEVIATIONS, INTERCORRELATIONS 
AND CROSS VALIDITIES FOR SAMPLE TWO IN THE 
FORM C STUDY OF THE BIOGRAPHICAL INVENTORY* 

(N = 379) 



‘o. 




1 


2 


3 


4 


5 


6 


7 


Means 


S. D. 




SAMPLE TWO CRITERIA 




















1 


Quantity of Work 


— — 














9.01 


2.64 


2 


Skill with People 


62 


— 












9.03 


2.47 


3 


Creativity of Work 


64 


51 


ao 










8.4-3 


2.45 


4 


Overall Work Performance 


78 


69 


80 


— 








9.20 


2.33 


5 


Patents 


05 


02 


14 


10 


— 






0.38 


1.20 


6 


Publications 


15 


14 


21 


21 


22 






6.62 


7.42 


7 


G. S. Level 


25 


30 


36 


34 


19 


62 


— 


11.81 


2.26 




SAMPLE ONE KEYS (Form C Study) 




















8 


Quantity of Work Total Score 


31 


21 


40 


37 


26 


35 


38 






9 


Skill with People Total Score 


30 


24 


36 


35 


21 


40 


45 






LO 


Creativity of Work Total Score 


27 


17 


4l 


36 


23 


31 


34 






11 


Overall Work Performance 


30 


20 


40 


38 


24 


35 


38 






L2 


Patents Total Score 


09 


-02 


27 


18 


24 


32 


29 






13 


Publications Total Score 


17 


20 


27 


27 


23 


66 


69 






*4 


G. S. Level Total Score 


16 


21 


29 


25 


19 


59 


72 








OTHER PREDICTORS 




















15 


General Intellectuality 


10 


07 


24 


19 


18 


20 


18 






16 


Independence 


10 


04 


27 


16 


20 


17 


13 






17 


Professional Self-confidence 


31 


18 


40 


38 


19 


31 


31 






18 


Miscellaneous 


11 


16 


14 


19 


10 


28 


28 






19 


Form A Total Score 


23 


13 


39 


32 


23 


29 


27 






£0 


Form B Total Score 


16 


12 


28 


23 


13 


16 


18 







* Decimal points omitted. 

I 
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.10 
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.01 



f 
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studios 9 what the scientist had accomplished in terms of publications and 
in tonns of the level of achievement that he had obtained within the 

organization* Or alternately stated, the ratings in the present study had 

less halo and were more independent of" other kinds of performancei thus 

they may reflect a somewhat different conception concerning creativity 

than evidently was the case in the previous studies. Also, it may be that 

people were promoted at the other centers more in terms of their creativity 

than at this one* 

The cross validity coefficients of the various new empirical keys in 
Sample One w- ye as follows: The Quantity of Work Key against the Quantity 

of Work Criterion *335 the Skill with People Key against the Skill wj.th 
People Criterion .365 the Creativity Key against the Creativity Criterion 
*4lj the Overall Work Performance Key against the Overall Work Performance 
Criterion .375 the Patents Key against the Patents Criterion .1?; the 
Publications Key against the Publications Criterion *59 3 the G* S« Level 
Key against the G* S* Level Criterion *69* 

The keys derived from the previous studios, while generally producing 
significant correlations, were nevertheless somewhat low<*r than what had 
been obtained previously* For example, in Sample One the Form A General 
„ Intellectuality Key correlated only *17 with Creativity? the Independence 
Key with the Creativity Criterion, *28? and the Professional Self-confidence 
Key with the Creativity Criterion, .41. 

For Sample Two * the cross validity coefficients were very similar* 

The now Form C Quantity of Work Key cross validated against the Quantity 
of Work Criterion, *31 5 "the Skill with People Key against the Skill with 
People Criterion, .24? the Creativity Key against the Creativity Criterion, 
.41? the Overall Work Performance Key against the Overall Work Performance 
Criterion, .385 the Patents Key against the Patents Criterion, .24? the 
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Publications Key against the Publications Criterion, .66; and the G. S. 

Level Key against the G. S. Level Criterion, .72. 

The magnitude of the cross validities on both samples against the 
last two criteria of Publications and G. S* Level is extremely high and 

certainly worthy of special notice. 

The results obtained from applying the keys from the previous studies 
to the scientists in Sample Two of the Form C Study were similar to those 
obtained with Sample One, Professional Self-confidence again being the 
most valid scoring key. It is significant to note, however, that the 
Professional Self-confidence Key yielded a level of prediction which was 
very comparable to the Form C Creativity Key which was constructed 
specifically to predict the Creativity Criterion at this research instal- 
lation. There has been a consistent trend across these studies for the 
keys constructed in the Form A Study (especially for the Professional 
Self-confidence Key, but also for the Form A Total Score as well) to 
correlate with the criteria at a different NASA research center as highly 
as the keys constructed specifically to, predict those criteria. 

Subsample Analyses . Since this was the largest sample of scientists 
yet studied and since the scientists at this research center are engaged in 
widely ranging types of research problems, a different type of analysis 
involving an organizational breakdown into various sub-samples of more 
homogeneous performance areas was carried out for the Form C Study. The 
scientists were placed into subgroups according to two criteria of 
classifications the area of research performance and the confidence 
expressed by supervisors in giving their ratings. The resulting five 
subgroups with a brief description of each are listed in Table 7* It is 
apparent from inspection of groups one, two, and three, that this subgrouping 
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has resulted in three quite separate areas of res oar ca performance. 
Scientists who were engaged only in flu* cl «nd flight mechanics were drawn 
from each of the first three groups and placed into group four, which is 
therefore the most narrowly defined group. Those in group five were so 
placed because, in contrast to those not placed in this group, their 
supervisors were more confident about the accuracy of their appraisals 
of the scientists® performance. 

Each of the five subgroups was then analyzed separately according to 
the procedures followed in the previous studies which involve an item 



alternative analysis carriod out in 
two subsamples of each group. 



a double cross validation design on 



Table 7 

Organization Breakdown 
for the Form C Study 



Total Sample 
(N=379. 390) 



1, Group One 

- (N=L11, 102) 

2, Group Two 

(N=136, 139) 

3, Group Three 

(N=l4-3, 138 ) 

4, Group Four 

(N=175, 184) 



The Total Sample, as in the previous analyses. Cross- 
validity coefficients listed under this group are an 
average of 5, Sample Oiio 5 ’ and '-Sample Two,*' 



Scientists in this subgroup are engaged in Analysis and 
Computation, Instrument Research, and Theoretical Mechanics, 

Scientists in this subgroup are engaged in Applied Materials 
and Physics, Dynamic I, cads, and Structures Research* 



Scientists in this s 
Aero-Space Mechanics 



a 5 "''"'. : — o-To curv-gcd in**Aero— physics 
, and Full Scale Research 



9 



Scientists in this subgroup are engaged only in fluid 
and flight mechanics. 



5, Group Five 
(N=300, 323) 



Scientists were placed in this subgroup on the basis 
of the greater confidence expressed by their supervisors 



in giving their ratings c 

Results for Subsamples , The results of these analyses are presented 
in Table 8, Although few of the obtained relationships are significantly 
different from the Total Sample analysis and none is significantly different 
in the positive direction, there are several interesting findings. The 
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*—■ Table 8 

Average Cross' Validities for Appropriate Keys Across the Two 
Odd-Even Sub-samples for each Organizational Group and Each 
Criterion in the Form C Study of the Biographical Inventory 



Criteria 



Organizational 

Groups 


Sample 

Size 


Quantity of 
Work 


Skill with 
People 


Creativity 
of work 


Overall Work 
Performance 


Patents 


Publications 


6 

t-4 

• 

CO 

0 


1* 


Total Sample 


(N=379, 


390) 


32 


30 


4l 


38 


20 


62 


70 


.2* 


Group One 


(H=lll, 


102) 


ia 


28 


30 


28 


14 


39* 


60* 


3. 


Group Two 


(H=L36, 


139) 


22 


26 


34 


34 


16 


57 


62* 


4. 


Group Three 


(N=lA3, 


138) 


36 


21 


44 


39 


26 


52* 


54* 


5. 


Group Four 


(N=175, 


184) 


34 


34 


48 


42 


21 


59 


63* 


6. 


Group Five 


(N=300, 


323) 


34 


34 


44 


38 


23 


57 


63* 



♦Cross-validities which differ significantly from the Total Sample results 
at the *05 level 3 as computed from Fisher’s Z function* 

Notes decimal points are omitted* 

analyses of Groups One and Two resulted in lower cross validities than in 
the Total Sample for all seven criteria* Group Three was interesting 
because the cross validities were higher for Criteria one, three, and 
four (all supervisory ratings) and lower for the other four criteria, with 
the results for Number of Publications and G* S* Level being significantly 
lower from the T)tal Sample beyond the *05 level of confidence. The 
analysis of Group Four resulted in cross validities for each of the four 
supervisory rating criteria and the Number of Patents Criterion which 
were higher than those obtained in the Total Sample analysis* The cross 
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validities for this group on the other two criteria were lower than in the 
S>tal Sample, with the difference for tho G. 5, Level Criterion being 
significant* There was also an increase in the cross validities across 
three of the four supervisory ratings (the other cross validity being 
equal to that of the Total Sample) and for the Number of Patents Criterion* 
As in the Group Four analysis, the Cross validities for the Number of 
Publications and C$, S* Level Criteria were lower than in the Total Sample, 
the latter being significant# 

The rationale behind this type of analysis Was that supervise* y 
ratings should bo more reliable and predictable when the performances 
being rated are more alike in a sample than when many different types of 
performances are being rated in the same sample* Inspection of Table 4 
shows this to be the case with Group Four on Criteria one, two, three, ' 
and four (the supervisory ratings). Although none of these cross validities 
was significantly different from the 7’ tal Sample, each o* them was higher 
than those obtained in the Total Sample and a trend here is quite apparent. 
This agrees with our rationale because, it will be recalled, Group Four 
consists of only those scientists who are engaged in fluiJ and flight 
mechanics and thus their tasks and performances are more nearly alike 
than the other groups. 

There is also* the game kind of trend in Group Five where there is an 
increase in three of the four crose validities (the other being equal 
to the Tbtal Sample) with the supervisory ratings, This was also expected 
since those placed in Group Five were scientists for whom raters were more 
confident of their ratings and presumably these ratings were more reliable 
and more valid than those for the other groups. 

It should also be noted that each of the group® s cross validities 
on the G. S» Level Criterion was significantly lower (at the ,05 level) 
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than that for the Total Sample. Apparently the magnitude of this relation** 
ship depends more upon the number (perhaps the variety ) ^of subjects than 
upon homogeneity of the performance tasks. The same can be said for the 
Number of Publications Criterion since all of the groups 5 cross validities 
were lower than that for the Total Sample, with those differences for 
Groups Or e and Three being significant. 

Summary of the Form C Study . The results obtained in the Form C Study 
were among the best in certain areas and in other areas among the worst 
(though still clearly significant) that had been obtained across the three 
NASA centers. The cross validity coefficients of .59 and .66 in predicting 
Publications represent a remarkable degree of prediction as do the cross 
validities of .69 and .72 in predicting G. S. Level. On the other hand, 
the cross validity coefficients in predicting the various supervisory 
ratings were less satisfactory than what had been obtained previously. 
However, the cross validity coefficients were all significant far beyond 
the .01 level of confidence and these result^ if used in a selection 
situation, could still make an important contribution to the identification 
of scientific talent. 
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Chapter 5 
57 

The Form C-l Study of the Biographical Inventory 

of the previous studios with the BI have been concurrent 
validity studies , providing some indication but little actual information 
relative to its validity in a predictive (longitudinal) situation. Therefore 
the present study was undertaken to determine how well the BI would perform 
in a follow-up study. 

Although little has been done in the way of comparison between 
concurrent and predictive validities for biographical inventories, a study 
was done on aptitude-like tests in which the concurrent validities were 
always higher, most of the differences ranging from ,06 to ,09 (Alf, 

19^3)* These results led to the conclusion that for his type of test, 
a shrinkage of at least .05 could be expected in switching from a concurrent 
to a predictive validity study. 

AT f ° s study, however, was designed in a way that would lead to 
obtaining similar results in both the concurrent and predictive validities. 
The same sample vras studied, being tested before starting school and then 
retested at the later time when the criterion of school -grades became 
available. Even though the criterion was the simpler one of school grades, 
none of the validities were nearly as high as the concurrent validities 
which have been reported for the BI. Thus, with the identical criterion 
on the same sample, it would be expected that differences in validity 
would sliest be minimized,, nonetheless, there were consistent differences 
always in favor of the concurrent validities. 

Description and Procedure of the Form C-l Stud y. Each of the NASA 
research centers participated in the Form C-l study by having all of 
their newly hired research professionals complete Form C-l of the BI as 
they reported for work, and just over a year later criterion data were 
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obtained on the performance of the research professionals. Form C-l is 
identical to the J00 item Form C, with the Form C-l designation being used 
here to differentiate between the two re spec tiv studies. 

Since the method of administering the BI was carried out over a time 
period of a few months , a psychologist from the University of Utah did 
not visit the NASA centers to help the personnel directors of the various 
centers with the administration of the inventory. For the first time, 
also, criterion data were collected entirely by personnel within the official 
system, not by visiting researchers who more obviously were collecting’ 
the data to take elsewhere for research purposes only. The extent to which 
this procedure may have fostered an attitude that was less cooperative 
than what might have been obtained if a psychologist had visited the centers 
to inform and explain the purposes and procedures of the research is not 
well known. This may have been especially important in the case of the 
supervisory ratings, since in the nrevious studies the visiting psychologist 
spent some time with most of the participating supervisors describing the 
importance of and the procedures involved in obtai *"ig accurate personnel 
evaluations. In a previous study (Sessions & Taylor, 1961), evidence was 
presented that personnel in the official system could administer tests at 
least equally well as visiting researchers, but the criterion data collected 
by members of the official system were not as good as those collected by 
visiting investigators which moro obviously was to be used for research 
purposes only. 

The procedures followed in the adminis.tration of Form C-l were 
identical to those used throughout all of the former studies, except for 
the instructions. In contrast to the previous studies in which the 
scientists were told that all data collected were for research purposes 
only, the scientists participating in this study were informed that while 
their specific answers to each question would be kept strictly confidential, 
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the scoring of the BI would be related to their future work performance. 

Only afterwards were they informed that the data- would be used for research 
purposes only. The Intent in using those instructions was to make the 
administration of Form 0-1 of the BI more comparable to an actual hiring 
situation and still maintain the cooperation of all those involved. 

No performance measures such as existing ratings, number of 
publications, or G. S. Level, which had been obtained in the previous 
studies, were collected for the scientists at any of the centers engaged 
in the Form C-l study. Thus, the only criteria collected were ratings on 
scales constructed by the investigators and completed by each scientists’ 
supervisor after each scientist had been on the job just over a year. 

These scales (aaraples of 1 which are presented in the Appendix) consisted of 
three supervisory ratings s one on Overall Work Performance, one on 
Creativity of Work, and a completed Creativity Check List, from which two 
scores were derived, namely a Creativity Score and a measure of Likeability. 

Also a fifth criterion measure was obtained by converting both the Creativity 
Rating score and the Creativity score from the Check List to standard 
scores at each of the centers and obtaining the arithnotice«l mean of 
for each scientist. 

It can be noted here that the time period covered by the criterion 
performance measures in this study is extremely short compared to that 
covered by the performances measures in previous studies. This could lead 
to a tendency for the supervisors in this study to rate the scientists more 
on likeableness initially than would be true later when the scientists’ 
actual productivity and creativity would become more apparent through their 
job performances and products. The extent to which this factor can influence 
the accuracy and reliability of performance measures is indicated in later 
sections of this report where the results of the several analyses are discussed. 
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Complete data on the Biographical Inventory, the two Supervisory 
Ratings, andtho Check List were obtained for 622 scientists from six 
different research centers, A return of this magnitude is truly out- 
standing since a researcher from the University of Utah did not participate 
in the collection of the data, thus illustrating once again the excellent 
cooperation of the NASA personnel in providing such returns by mail. 

Two kinds- of analyses were carried out in the Form C-l study, both 
utilizing high speed computer techniques. In the first analysis, scoring 
keys derived from previous studies were applied to the BI responses of each 
scientist. The resulting scores were then correlated with the criterion 
measures to obtain cross validity coefficients for each of the centers and 
for the total sample. The second kind of procedure involved an item 
alternative analysis on the total sample carried out in a double cross 
validation design (the same procedure used in the former studies). 

Scoring Kov Analysis in the Form C-l Study, Seven previously developed 
keys were used in scoring the BI responses. Since an elaborate description 
of their content and development is presented earlier in thl’s 
report as discussed in the Form A and Form B studies and in Ellison 0*964) , 
only a brief account will be given here. The keys used weres Creativity I 
(Minimum Weight) , Form A Total Score, Creativity. II (Maximum Weight or 
Professional Self-confidence), Modal Response, Off-Line, False Modesty, 
and Exaggeration, In general, the keys were developed by examining the 
item alternative analyses of the previous studies and retaining those item 
alternatives which resulted in a cross validity coefficient of at least 
•20 with the appropriate criterion and to which at least 5 per cent of 
the sample responded. The Creativity I or Creativity Minimum Weight Key 
was constructed in the above manner based on the Creativity Criterion 
measures in the previous studies. It differs from the Creativity II Key in 
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that the minimum validity cut-off level for the retention of item alternatives 
was lowered so that more item alternatives were scored. Also the Creativity 
II Key was originally an at priori key which was refxned through empirical 
analyses* The Form A Total Score Key was derived from the procedure 
described above in the Form A study of the Biographical Inventory. 

The remaining four keys are correction scores designed to adjust 
distortions in the BI responses. That is, the Modal Response Key was 
designed to account for the BI scores of those scientists who tend to 
respond in the particular modal range of the various BI items, and thus 
their responses do not reflect an accurate BI score in relation to their 
criterion score. The Off-line Key was designed to account for the degree 
of error (regardless of direction) that occurred in trying to predict the 
criterion distribution from the distribution of the BI predictor scores for 
a sample of scientists. This Off-line Key thus was an attempt to identify 
those scientists who, in’ psychometric language, would be called false 
positives-*- those who had a BI predictor score larger than their criterion 
score, and false negatives— those who had a predictor score lower than their 
criterion score. Finally, the Exaggeration Key and the False Modesty Key 
were derived from the Off-line Key. Those scoring high on the Exaggeration 
Key consisting of scientists identified as false positives— their self 
descriptions being somewhat inflated as judged in light of their criterion 
measures. Those scoring high on the False Modesty Key were identified as 
false negatives— those scientists who were conservative and tended to 
describe themselves, their backgrounds, and experiences in an overly 
modest and humble fashion in relation to their actual performance. 
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V/ith seven different ere die tor and correction score keys, various 
combinations can be utilized in partial and multiple correlational methods 
for maximum prediction of the criteria as presented in Guilford (195*0* A 
procedure of this nature should result in greater determination of the dimen- 
sions involved in the variables under investigation. 

The data analysis for this part of the Form C-l study was completed 
at the Western Data Processing Center at Los Angeles, California and was 
carried out with high speed electronic computers. The computer program 
scored each subjects® BI responses according to the seven above mentioned 
keys, correlated each resulting score distribution with the criterion 
measures, intercorr elated the criterion scores and also intercorrelated 
the keyed scores. Needless to say, this procedure added a great deal to 
the speed and overall efficiency in determining the results of this study. 
Results of the Combined Sample Analysis (Sample I) . The procedure 
described above was first applied to the combined sample of 622 scientists 
from six different research centers. Combining the scientists for analysis 
in this manner was justified since computation indicated no significant 
differences from center to center in all of the mean criterion scores and 

all seven of the mean keyed scores. There is also some justification 
for this from previous comparable results across centers indicating that 
if results obtained here were again positive, this x«mld suggest broad 
applicability of the scoring keys and the research procedures. 

The results of the Combined Sample Analysis are presented in Table 9* 

The first three variables listed are criterion measures. These three 
criteria were used in the Total Sample Analysis because these were the 
three measures common to all six research centers involved. That is, 
certain criterion scores were obtained at several of the centers which 
w?re not obtained at others. The interrelationships between these three 
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Table 9 

CRITERION INTERCORRELATIONS , KEY SCORE INTERCORRELATIONS , MEANS, 
STANDARD DEVIATIONS, AND CROSS VALIDITIES FOR THE TOTAL SAMPLE 

(SAMPLE I) IN THE FORM C-l STUDY* 

(N = 622 NASA SCIENTISTS) 







1 2 3 4 


5 6 7 8 9 10 


Means 


S. D. \i 


1. 


Overall Work Performance 


— 




9.56 


2.06 


2 . 


Creativity Rating 


71 ~ 




8.61 


2.28 | 


3. 


Creativity Check List 


69 74 — 




49.49 


10.26 


4. 


Creativity 1 Key 


09 15 17 — 




98.79 


7.46 


5. 


Form A Total Score Key 


13 18 20 88 


W tM 


108. 30 


10.49 


6. 


Creativity 2 Key 
(Prof. Self-Conf.) 


12 17 20 72 


85 - 


100.36 


5.40 

.'.S 


7. 


Modal Response Key 


07 04 05 -14 


10 -09 — 


109.14 


2.71 


8 . 


Off-Line Key 


-04 -06 -10 -45 


-21 -43 -01 — 


100.38 


3.82 


9. 


False Modesty Key 


.06 -06 -12 -29 


-13 -32 -15 84 — 


104.22 


2.33 


10 . 


Exaggeration Key 


-02 01 02 48 


25 43 -14 -78 -38 - 


. 96.08 


2.35 




Decimal points omitted. 


r ~ .08; r ^ 

.05 .01 


= .11 
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universally used measures are quite high, generally exhibiting more common 
variance than had the criterion scores in the previous studies of the 
biographical inventory, possibly due to the shorter time on the job and 
the less differences across performances yet noted by supervisors. 

Variables 4, 5» and 6 are predictor keys developed from previous 
studies so that, as well as for the last four correction keys, the 
correlations between these measures and the criteria are all cross 
validity coefficients. The Creativity I Key correlated .09 with the 
Overall Performance Criterion, .15 with the Creativity Rating, and .17 
with the Creativity Check List. As expected, this key was more effective 
in predicting the Creativity Criteria than other types of performance. 

The Form A Total Score Key correlated .13 with the Overall Performance 
Criterion, .18 with the Creativity Rating, and .20 with the- Creativity 
Check List. Again the Creativity Criteria were predicted to a higher 
degree than was the Overall Performance Criterion, but there was some 
increase in its prediction with this Form A Key over that found for the 
Creativity I Key. The Professional Self-confidence Key (Creativity 2) 
exhibited about the same pattern, correlating .12 with the Overall Performance 
Criterion, .17 with the Creativity Rating, and .20 with the Creativity 
Check List. The key scores were quite highly interrelated, the inter- 
correlations ranging from .72 to .88. The validities reported here are 
much lower than those obtained in the predicting of similar criteria in 
previous studies, while the key score intercorrelations here are somewhat 
higher. 

Although eig-ht out of nine of these cross validities are significant 
beyond the .01 level of confidence, the magnitude of these relationships 
are considerably lower than what had been obtained in previous studies. 

Some decrease could be expected in a follow-up study of this nature, but 
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somewhat better results wore expected since those predictor keys had held 
up very well In previous studies , even from one center to another. There 
are several possible explanations for these results, perhaps the most valid 
of which concerns the criterion measures. This avenue of explanation will 
be referred to throughout this report when relevant evidence Is discussed , 
and a synthesis will be attempted along with other considerations when 
all the evidence has been presented. 

Variables 7 through 10, the correction keys, did not perform as well 

as was expected, These keys were constructed to perform either as 

» 

suppressors or to identify any possible additional criterion variance not 
accounted for by the predictor keys. In order to perform this function, 
the correction key must either act as a suppressor or must itself contribute 
something to the prediction of the criterion. Inspection of Table 5 shows 
this to be the case with Variable 7, the Modal Response Key, which has 
positive correlations with all three criteria and negative relationships 
with variables h and 6, which are predictor keys. Variable 8, the Off-line 
Key, correlated negatively with all three criteria and moderately high 
and negatively with the predictor keys. Variables 9 and 10 resulted in 
essentially the same pattom since they were constructed from the Off-line 
Measure. 

The Modal Response Key was combined with the Professional Self- 
confidence Key in multiple predictions of the three criteria to illustrate 
the effects of the correction scores. The results (presented in Table 10) 
were as follows: the multiple prediction of the Overall Work Performance 

Criterion was .15* that for the Creativity Rating, .18, and for the 
Creativity Check List, .31. Each of these coefficients, although relatively 
small, show some improvement over the Professional Self-confidence Key when 
used alone. With higher first order validities for the predictor keys these 
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Table 10 

Maximum Multiple Prediction of the Criteria 
In the Sample I Analysis* 



Predictors 


1 


2 


3 


Modal Response Key (Variable 7) 


.15 


.18 


• 21 


and Creativity 2 Key (Variable 3) 









* r = •1°3 r = *13 
. *05 .01 



correction measures would have made a more important contribution. 

In view of these results, and since the Total Sample Analysis was 
composed of scientists from six different research centers, it was decided 
that each center would be studied separately. This kind of analysis was 
undertaken in order to obtain greater insight into the above relationships 9 
that is, to determine whether the BI would more accurately predict the 
criteria at some centers than at others and to discover further relation, 
ships which would help to more fully understand the predictive power of the BI. 

Results of the Sample II Analysis . A total of 269 scientists who had 
completed the BI and on whom complete criterion information had been 
obtained were included in the Sample II study. 

Five criterion measures were obtained for this analysis. Three of 
these, Overall Work Performance, Creativity Rating, and the Creativity 
Check List, are the same criteria used in the Combined Sample Analysis. 

From the Check List, a control variable called “Like ability” was obtained 

♦Although other combinations of the BI keys could have been utilized 
in multiple prediction, the particular combination of Variable 7 with 
Variable 6 resulted in the Maximum prediction of the criteria in the 
Sample I analysis, and therefore is the only combination presented. This 
samo procedure of key selection for maximum multiple or partial correlation 
will be utilized in similar tables throughout this report. 
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in order to determine to what extent tnis kind of factor might influence 
the supervisor ratings. This likeability variable was constructed from 

two statements in the Creativity Check List having to do with whether the 

scientist was pleasant to work with and how well he worked with his 

supervisor. It was felt that this procedure would be a useful indication 

of the extent to which this Likeability factor was influencing the supervisors 

in their performance appraisals, since they were supposed to be considering 

only creative performances in the completion of the Check List. It was 

intended that, if common variance was found between this score and the 

criteria, the Likeability measure would be partiaHed out or combined 

with a BI key to more fully determine the dimensions and predictability of 

the criterion scores. 

Since It was found in the Combined Sample Analysis that the two 
creativity criteria had a great deal of variance in common but that neither 
was as predictable as expected, a fifth criterion involving these two 
measures was constructed which was thought to be somewhat more accurate 
and predictable. This was accomplished by menas of a special computer 
program which first converted the Creativity Rating and the Creativity 
Check List measure for each scientist to a standard score and then computed 
a mean of the resulting two scores. This procedure was followed for 
each center separately. The resulting standard scores therefore were 
based upon each scientist’s own group. 



The results of the Sample II analysis are presented in Table 11. 

Variables 1 through 5 are the criteria as described above. As in the 
Combined Sample Analysis, the first three criteria are characterized by 
quite high intercorrelations (though not very high if they are considered 
to be reliability estimates). Variable 4, the Likeability score, correlated 
with all four of the other criteria to a significant degree. Thus, as 
expected, this dimension was an influencing factor in the supervisory 
ratings. This was especially apparent with Variable 1, where the 
correlation was .37* 

Variables 6 through 12 are the BI keys, identical to those described 
above as used in the Combined Sample Analysis and throughout the Form C-l 
study. Variables 6 through 8, the predictor keys, show about the same 
prediction pattern with the criteria as was shown in the Sample I analysis. 
As was expected, all three keys were somewhat more successful in their 
prediction of Variable 5 > the combined Criterion, than with either 
Criterion Variables 2 and 3 taken alone. Variable 9, the Modal Response 
Key, again seems to be identifying some additional criterion variance 
other than that identified by the predictor keys. Again Variable 10, the 
Off-line Key, and its derivatives, Variables 11 and 12, (the False Modesty 
and Exaggeration Keys) did not perform as expected, since their consideration 
does not result in increased prediction of the Creativity or Overall 
Performance Criteria. 

Examination of Variable 4 and its relationship with all eleven of 
the other variables makes apparent a very interesting trend. As was 
mentioned previously, this Likeability measure correlated significantly 
positive with all four of the other criteria scores. This variable also 
correlates positively (although not significantly) with all four of the 
correction keys, but has slight negative relationships with the three 
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predictor keys* These findings give further evidence that, not only was 
this Likeability characteristic an influencing factor in the supervisory 
ratings, but it also affected the predictive power of the BI in its 
relationships with the Creativity and Overall Performance Criteria. 

These relationships can be further investigated utilizing the procedures 
for partial and multiple correlation as prerented in Table 12. 

When the Likeability measure (Variable 4) is held constant, the 
correlation between the Form A Total Score Key (Variable 7) and the 
Overall Work Performance Criterion (Variable l) is raised from .13 to .17 
and that between Variable 7 and the Combined Criterion (Variable 5) 
is raised from .23 to .26. The multiple prediction of Criterion 1 with 
Variables 4 and 7 is .40 and the multiple prediction of Criterion 5 with 
these two variables is .34. This latter procedure is not fully justified, 
since the Likeability score is a part of the other criterion scores and 
not a predictive measure. A more vfilid procedure which has not yet 
been developed would be to construct a HI key to predict Likeability and 
then determine its relationships with these variables. However, the 
procedure was carried out here to illustrate the degree of influence which 
this factor had in the criterion measures and the magnitude of correlation 
which could be obtained with its use. Use of this technique also has 
some justification since the Likeability score is overlapping a portion 
of the criteria (Variables 1 and 5) predictor (Variable 7) is 

overlapping the non-like ability part of the criteria. 

It thus seems apparent that this Likeability dimension was indeed 
one of the difficulties affecting the criterion measures obtained. Although 
the raters were instructed to consider only the creativity of the ratee, 
they were obviously influenced by the Likeability dimension, and since 
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Table 12 

Maximum Multiple and Partial Prediction of 
The Criteria in the Sample II Analysis 



Predictors 


Partial Correlations 

Criteria 

1 


2 


Likeability (Variable 4) and 


Form A 




Total Score Key (Variable 7) 


.17 


.26 




Multiple Correlations 




Variable 4 and Variable 7 


.40 


.34 



r ,05 = - 155 r .01 = - 19 



the BI keys were constructed without taking this into account, this 
situation resulted in a detrimental effect in the predictability of the 
criteria with the BI keys. However, a correlation of *40 in predicting 
Overall Work Performance and .34 in predicting Creativity, which results 
from these dimensions taken together, is quite a noticeable degree of 
prediction in light of other potential contaminators and also probable 
unreliabilities in the criteria. 

Results of the Sample III Analysis . Complete data were obtained on 
a total of 56 scientists who comprised the sample in this study. 

The results of the Sample III analysis are presented in Table 13* 

Variables 1 through 5> the Criterion measures, are identical to those 
described in the Sample II analysis. The interrelationships among the 
criteria in this sample (with the exclusion of the Likeability score) 
are lower than those for the Total Sample or for any other center studied 
separately. The Likeability score, although not significantly related to Criterio: 
1, is again observed to be a significant factor in the creativity criteria. 
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The predictor keys, Variables 6 through 8, are again characterized by 
high interrelationships, but of the fifteen cross validities between these 
predictors and the five criteria, not one is significantly different from 
zero* The Correction Keys, Variables 9 through 12, also added no significant 
variance, generally exhibiting negative correlations with the criteria. 

It can thus be observed thaTi not only are the criteria characterized by low 
interrelationships, but nothing else (save Likeability) is related to these 
criteria either. Needless to say, this sample added little to the 
effectiveness of the BI in predicting the criteria in the Combined Sample 
Analysis or to the understanding of the prediction phenomena. 

Results of the Sample IV Analysis , The Sample IV analysis was composed 

of scientists from two different research centers, the two centers being 
combined because complete data were obtained on only 22 scientists at 

one of the centers and only 35 at the other; thus, the two were combined 
to make a sample of 57 scientists. 

The results of the Sample IV analysis are presented in Table 14. 

Variables 1, 2, and 5 (the criteria) are characterized by high inter- 
correlations, but Variable 4, the Likeability score, was not significantly 
related to any of these criterion measures. This latter finding is quite 
significant, since this was the first sample in which such low 
relationships were found between Likeability and the other criteria. Such 
a finding indicates that the supervisors in this sample were not 
significantly influenced by this dimension in their performance ajpprasials , and 
thus their ratings should represent a much clearer picture of these 
scientists 9 performance than do those ratings obtained at the other 
centers. 

Variables 6 through 8, the predictor keys, were quite successful in 
their prediction of the criteria in this sample. All twelve of the 
cross-validities between these predictors and Criteria 1, 2, 3t and 5 
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are significant, nine of them beyond the ,01 level. The Overall Work 
Performance criterion seems to be the most predictable, its correlation 
with the Total Score Key being .45* The Creativity Chock List was also 
very highly predictable, its correlation with both the Total Score Key 
and the Creativity 2 Key being .44. The Creativity Rating was also 
predicted to a magnitude that is significantly different from zero at the 
.01 level by these latter two keys. The correlations between these three 
predictor keys and the Likeability score were all negative, though essentially 
zero 5 thus, Likeability was apparently not a significant factor influencing 

the .criteria or the BI predictions. 

The Correction Keys were also very effective in their performance 
with this sample. The Modal Response Key is identifying additional 
criterion variance since it correlates .13 with Variable 1 and -.20 with 
Variable 8 . The False Modesty Key is also effective since its correlation 
with Variable 1 is .17, with Variable 8 is -.63, and with Variable 9 
is -. 11 . 

The partial and multiple prediction of the Overall Work Performance 
Criterion and a Creativity Criterion (the Creativity Check List), are 
presented in Table 15, When the Likeability variable is held constant, 
the prediction of Criterion 1 by the Form A Total Score Key is .46 and 
the prediction of the Creativity Criterion (Variable 3 ) by the Creativity 
2 Key is .43. The multiple prediction of Criterion 1 with the Total 
Score Key, the Modal Response Key and the False Modesty Key is .55 and 
that of Criterion 3 with the Creativity 2 Key and the Off-line Key is 
. 51 . These correlations between predictors and criteria are the highest 
obt a ined in the entire Form C— 1 study and their magnitude indicates a 
truly remarkable degree of predictive validity for initial job performance* 
This becomes especially apparent when one realizes the tremendous 
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difficulty involved in the prediction over a period of time of such complex 
and sometimes subtile criteria. 

Table 15 

Maximum Multiple and Partial Prediction of 
the Criteria in the Sample IV Analysis 



Partial Correlations 

Criteria 

Predictors 1 2 

Likeability (Variable 4) 

and Form A Total Score Key .46 

(Variable 7) 

Likeability (Variable 4) .45 

and Creativity 2 Koy (Variable 8) 

Multiple Correlations 

Total Score Key (Variable 7) 

Modal Response Key (Variable 9) , and *55 

False Modesty Key (Variable 11) 

Creativity 2 Key (Variable 8) 

and Off-line Key (Variable 10) *51 



r 



05 



- .32$ r = .39 

.01 



Results of the -Sample V Analysis . Complete criterion and biographical 
data were obtained on a total of 92 scientists who comprised this sample. 

The results of the Sample V analysis are presented in Table 16. The 
Overall Work Performance and the Creativity Criteria are again highly 
inter correlated, and the Likeability score is again observed to be a 
significant dimension influencing the criterion measures. None of the 
fifteen cross validities between the predictor keys (Variables 6 through 8) 
and the criteria is significantly different from zero. Of the four 
correction keys (Variables 9 through 12), the Modal Response Key seems 
to be the only one contributing to the prediction of the criteria. In 
fact, this correction key is more highly related to each of the criteria 











Table 16 

CRITERION INTERCORRELATIONS, KEY SCORE INTERCOHR3LATIONS, MEANS, 
STANDARD DEVIATIONS, AND CROSS VALIDITIES FOR FORM C-l SAMPLE V 

(N = 92) 





n 



than is any other key in the analysis. Its correlation with Criterion 1 
is .1ft with Criterion 2, .16, with Criterion 3> .23* ^ Criterion 5> 

.20. These latter two are significant at or beyond the .05 level. 

The results of this analysis seems to indicate that among other things 
we are once again dealing with contaminated and unreliable criteria, since 
nothing semms to be related to these criterion measures except Likeability 
and Modal Response. The BI was therefore relatively unsuccessful for the following 
prediction of criteria in this sample, and like Sample III, this sample 
added little to the effectiveness of the BI in predicting the criteria 
in the Total Sample Analysis , 

Results of the Sample VI Analysis. This final sample included in the 
Form C-l study was comprised of 149 scientists on whom complete data were 
obtained. 

The results of the Sample VI analysis are presented in Table 17# It 
can be observed that the criteria (Variables 1 through 7) are different 
from those used in the previous analyses. Variables 1 and 2 are identical 
to those obtained in the other samples, but the other five are new. The 
creativity score obtained for Variable 3 came from a longer form of the 
check list than that used in the previous analyses (this one having 78 
items, the former having 51 items). The remaining criterion scores, 

Variables 4 through 7, all came from this Creativity Check List. Variable 
4 was obtained by simply counting the total number of items which a ra^er 
had checked for each scientist and including this number as his score. 

Variable 5 is a ratio score obtained by counting the number of positive 
items checked anti dividing this sum by the total number of items checked. 
Variables 6 and 7 were obtained as the result of a quite lengthy procedure 



as described bolow. 



Table 17 

CRITERION INTERCORRELATIONS, KEY SCORE INTERCORRELATIONS, MEANS, 
STANDARD DEVIATIONS, AND CROSS VALIDITIES FOR FORM C-l SAMPLE VI 

(N = 149) 
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The modifications described in the Check List which resulted in 
Variables 4 and 5 were carried out because it was felt that the Check 
List was perhaps our best instrument for getting a criterion score on 
creativity. This consideration further resulted in the derivation of 
Variables 6 and 7, obtained through a very novel procedure designed to 
fully maximize the potentialities of the instrument and to provide a more 
reliable and predictable criterion target. The procedure involved an item 
alternative analysis of the 78 statement Check List against seven other 
measurers;* 2 criterion scores and 5 BX key scores. These were; (l) the 
Overall Work Performance Criterion; (2) the Creativity Eating Criterion; 

(3) the Form A Total Score Key; (4) and (5) two forms of the Off-line 
Key 9 (6) the Creativity 1 Key, and (7) the Creativity 2 Key (Professional 
Self-confidence). The results of this analysis thus led to the determination 
of the relationships which each item in the Check List had with the other 
seven measures and to determine which items were w live M and which were 
M dead 5! weight in respect to each variable. Needless to say, this analysis 
will be extremely useful in any future work with this criterion form. 

Through the examination of these results, two scoring keys were developed 
for the Check List; one based upon the intercorrelations between the Check 
List statements and the two criterion scores and the other based upon 
the intercorrelations between the Check List statements and the creativity 
keys. The former was comprised of 39 statements, sixteen statements having 
the highest average negative correlations with the two criteria and twenty- 
three statements having the highest average positive correlations. This 
key, when applied to the Check Lists obtained for each of the 149 scientists 
in the sample, resulted in scores which comprise Variable 6. The same 
procedure was carried out for the second key and resulted in twenty- 
five statements, twelve negative and thirteen positive, each having the 
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highest average intorcorrelations with the two BI creativity keys# This 
was then applied to each scientist’s Check List , the resulting scores 
comprising Variable 7, which thus is a criterion score built to match a 
predictor score# 

Variables 8 through 10, the Predictor Keys, and Variables 11 through 
l4, the Correction Keys, are identical to those used in the previous samples# 

It can be observed from Table 13 that the first three criteria, Variables 
1, 2, and 3 are again characterized by high intercorrelations# Variable 4 
shows a significant relationship with Variable 3* 'the Creativity Check 
List, so it appears that the sheer number of items checked (both positive 
and negative) in the Check List does have an influence upon the criterion 
scores obtained with this measure# However, because of the relationships 
Variable 4 has with the BI keys, partialling out its effects does not 
result in an increase in the prediction of the Check List Criterion. It 
is also apparent from the criterion intercorrelations and cross validities 
with Variable 5 that little additional information was obtained with its 
inclusion over what was found from the original Check List Criterion# Thus 
although tho derivation of Variables 4 and 5 did not result in increased 
prediction of the Creativity Check List Criterion, Variables 6 and ? roveal 
significant validity information, as discussed below. 

The Predictor K eys , Variables 8 through 10, show significant validation 
results. Variable 8 exhibits cross validity coefficients which are 
significant at or beyond the .01 level for each of the three original 
criterion scores. Variable 9 was just about equally as successful, while 
Variable 10 was somewhat less so. As mentioned previously. Variable 4 has 
non-significant relationships with the BI keys, and the cross validities 
with Variable 5 are of the same magnitude as those with Variable 3 so that 
little additional information results from the inclusion of these two 
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criterion scores « The other two derivatives of the Check List, Variables 
6 and 7, did result in greater predictability of this dimension since the 
correlation between Variable 6 and Variable 3 is ,30 and that between 
Variable 7 and Variable 6 is .25. The same pattern of increased prediction 
is apparent with the other two predictors, Variables 10 and 11, where 
appreciable increases in the cross validity prediction of the Creativity 
Check List are observed* 

The Correction Keys, Variables 11 through 14, generally were unsuccessful 
since they added little or no additional criterion variance* However, the 
Off-line Key, Variable 12, was combined with the Creativity i Key for 
maximum prediction of Variables 1 and 2 as observed in Table 18* 

Table 18 

Maximum Multiple Prediction of the Criteria 
in the Sample VI Analysis 



Criteria 

■ . i ■ —— — —— 

Predictors 1 2 

Creativity 1 Key (Variable 8) 

and Off-line Key (Variable 12) .24 *25 



r= .20; r - *24 
.05 .01 

Summary and Discussion of the Key Score Analysis . It is now apparent 
that the breakdown of the Total Sample Analysis into separate analyses of 
the various centers has resulted in greater under standing of the relationships 
involvod. The significant but relatively low validities obtained in the 
combined sample can now be better understood in light of the results of 
the various subsample analyses. It will be recalled that about the same 
results were obtained with Sample II as with the Total Sample. Samples 
III and V resulted in essentially zero validities while Sample VI resulted 
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in higher validities than in the Total Sample and Sample IV resulted 
in comparatively high prediction of all the criteria. The task then ? is 
to examine these various results and discover evidence which will help to 
explain those differences in prediction from center to center. 

The most obvious explanation for the low validities obtained in the 
Total Sample Analysis is the extreme complexity involved in the combined 
study of 622 scientists from different working environments. This certainly 
suggests some interesting work climate differences existont in the different 
research organizations studied, and the further investigation of such 
differences might lead to more understanding of the results found in 
the Form C-l study. 

As has been mentioned throughout th?.s report, a major difficulty 
involved in this complexity lies in the criterion measures obtained from 
these different research centers. That these measures were not obtained 
under maximally desire able conditions and the observation that in some centers 
the criteria was much more predictable than in others (those centers 
having the smallest number of people to be rated contributing the most 
predictable criteria) suggests the unreliability and inaccuracy of these 
pooled supervisory ratings. Further evidence pertaining to this suggestion 
will be presented in the next section where the item analysis is discussed. 

Also in relation to this complexity, it can be observed that for 
those centers which showed poor validation results, the Likeability 
dimension was demonstrated to be more of a contaminating influence than 
for those with good validation results. This feature has been proven to 
be an especially important factor in the differences found from center to 
center where the Likeability dimension was observed to be a major influence 
in the criterion measures obtained at some of the centers and in the 
prediction of these measures with the BI. 
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Mention should also be made here concerning the internal consistency 
of the various BI keys in relation to each other. As expected , in almost 
every case the predictor keys woro negatively related to the correction 
keys, the latter usually showing some positive relation to the Likeability 
criterion. With this being the case in the better validation results, 
these measures were combined in effective multiple and partial correlational 
methods for maximum prediction of the various criteria. 

It is therefore concluded from these considerations that in spite 
of the tremendous complexity involved, the potential predictive power 
of the BI has been best indicated in those instances where more adequate 
criterion targets were available. Reference is made here to the Sample IV 
analysis, in which cruss validities of .55 and .51 were obtained, respectively, 
in the prediction of the Overall Work Performance and the Creativity Check 
List criteria. 

Item Alternative Analysis in the Form C-l Study. This analysis was 
undertaken to determine whether keys specifically constructed for the 
prediction of the criteria in this study would be more successful in 
validation results than were the keys developed from previous concurrent 
validity studies. It was thought that perhaps scientists who are already 
on the job (as was the case in previous studies) would respond somewhat 
differently in the BI than those who are applying for a position, which 
was the case in the present study. If this is so, then the relationships 
between the 31 responses and the criteria would also be affected. 

Therefore, insofar as the criteria are accurate and reliable, construction 
of new keys specifically for these responses should provide better 
validation results. 

The procedure involved in this analysis was essentially the same as 
that described for the item analyses in previous studies. The 622 scientists 
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were split into two Odd-Even samples of JOQ and respectively. An 

item alternative analysis was then carried out in a double cross-validation 
design by means of a high speed computer at the University of Utah Computer 
Center. The computer program utilized for this analysis was essentially 
the same as that used in former studies, except that io has been somewhat 
revised and expanded. 

As a result of this double cross validation design, the correlations 
reported in Tables 19 and 20 between the various keys and the criteria 
are cross validities. As these tables indicate, a separate key was developed 
for each of the four sections of the BI as well as for the total instrument, 



against each of the three criteria. Section I is concerned with 
Developmental History, Section II with Parents and Family Life, Section III 
with Academic Background, and Section IV with Adult Life and Interests. 
Table 15 gives the results of the keys developed on the Even Sample 



as applied to the Odd Sample. The most striking thing about these 
results is that, for all three of the criterion keys, the only section 
of the BI contributing validity significantly different from zero is 
Section IV, tho Adult Life and Interest Section. In each case, the 
validity coefficients exhibited by any of the other sections is essentially 
zero, thereby leaving Section IV the only valid part of each key. Thus, 
the correlation between Variable 7 and Variable 1 as .1^, between Variable 



12 and Variable 2, .18, and between Variable 17 and Variable 3» 

It will be noted that these cross validities, although rather low, are all 
significant at or beyond the .01 level of confidence. 

Another interesting feature of these results is that Variable 17 
exhibited higher cross validities (.22 and . 26 ) for the otnei two criteria 
(Variables 1 and 2) than any of the keys specifically constructed for the 



prediction of these two criteria. 
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Table 20 gives the results of the keys developed on the Odd Sample 
as Applied to the Even Sample* Although the criteria intercoyrelations are 
somewhat higher for this sample , these results show about the same pattern 
as those for the Odd Sample, Section IV exhibiting the highest correlations 
in each key* Variable 7, Section IV of the Overall Performance Key, 
exhibited the highest correlations with all three criteria^ higher than 
any of the keys specifically constructed for the prediction of the other 
criteria. Thus, the maximum prediction of the criteria as shown with this 
key are .15 for Variable 1, .17 for Variable 2, and .21 for Variable 3. 

The superior performance of Section IV in this analysis is entirely 
consistent with all of the former studies of the Biographical Inventory, 
where this section resulted in better prediction of the criteria than the 
other parts of the instrument. The difference between results in this 
and in the former studies lies in the rather low validities of all the 
BI sections, and especially the extremely low and sometimes negative 
validities of Sections I, II, and III. With this low magnitude for the first 
three sections, Section IV actually predicted the criteria to a higher 
degree than did total scores across all four sections. 

Sum mary and Discussion of the Item__Analygis^ It can thus be observed 
that when considering only the case of the Total Sample, the item analysis 
of the Form C~1 data resulted in somewhat better prediction of the criteria 
than had the application of previously constructed keys to these same 
criteria, although the increase was rather small and quite variable from 
the Odd to the Even Sample. Thus the maximum prediction of the Overall 
Work Performance Criterion in the Combined (Sample I) Key Score analysis 
was .15 while in the item analysis it was .19 (this latter figure 
obtained by averaging tho highest correlation between a key and this 
criterion in each of the Odd-Even Samples). For the Creativity Rating 
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Criterion, tbo key score analysis resulted in a prediction of .18, and 
for the item analysis.; a 22, and finally, for the Creativity Check List 
Criterion, . 21 and .23 respectively, for the two types of analyses. 

A methodological problem needs to be mentioned here. Because of the 
relative unreliability of the criteria and other measurement contaminants 
in this study, the particular type of biserial item analysis procedure 
utilized in this analysis resulted in only an average of 6 or 7 item 
alternatives being scored for each subject in the prediction of each 

criterion* When the shortness of these keys is taken into account, a cross 

* 1 

validity coefficient of *26 in predicting creativity is quite remarkable. 
However, had more items been utilized in these prediction s, the validities 
might have been increased. 

These results provide evidence in relation to one of the main questions 
under consideration in this analysis. That is, construction of new keys 
specifically designed to predict the criteria in this sample did not 
result in appreciable increases in prediction evor what had been obtained 
with the application of previously constructed keys to these same criteria. 
This indicates further evidence in support of the contention that the 
criteria under consideration in this study are jot adequate. This 
statement holds true for the total sample considered together, although 
more information could be obtained relative to these relationships if each 
center were subjected to separate item analyses. However, the statistical 
procedures involved depend upon a large number of subjects, which would not 
bo the case if each center were studied separately. Therefore, based upon 
the evidence now available, some stability and reliability of these 
previously constructed keys has beon demonstrated, even in the prediction 
of questionable criteria. 



90 



Concluding Remark s , The Biographical Inventory has now been studied under 
several concurrent validity settings, and, with the completion of the Form 
C-l study, it has also now been studied under predictive (longitudinal ) 
conditions. The proven effectiveness of the instrument in the former 
concurrent situations has held up quite well for some of the centers 
(Samples IV and VI) in the longitudinal design, and not so well for some 
of the others (Samples II, III, and V). Several possible explanations 
have been entertained in this report to account for these differing 
results. The following discussion brings several other relative considerations 
to light. 

The extreme complexity in such a study as reported here cannot be 
overemphasized. Studying so many scientists from different working 
environments and from different areas of interest introduce an especial 
strain upon the criterion problem, especially when pooling samples togehter 
across NASA centers. It has been shown in the Form C breakdown analysis 
that prediction increases when the subjects under investigation are 
working in more homogenous areas of endeavor. The same has been shown 
in the Form C-l study where increased prediction resulted in some cases 
when the different centers were studied separately. However, this is 
only one of the problems involved in this complexity % many others must 
remain unaccounted for. 

Another aspect of this problem is that there are undoubtedly many 
differences in the group of scientists involved in the Form C-l study and 
those who were involved in the previous investigations. One of these 
differences is the educational status of these groups. In the Form A study 
of the BI, 24 per cent of the scientists had a Bachelor’s degree or less; 

38 per cent had some graduate work, but no degree; 13 P er cent had a Master’s 
degree, l6 per cent had some graduate work beyond the Taster’s degree; and 
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9 per cent had the Ph.D. degree. The corresponding percentages in the Form 
C-l study were as follows! 68 per cent had a Bachelor 0 s degree or loss 5 
23 per cent had some graduate work, but no degree 5 6 per cent had a 
Master 5 s degree 5 2 per cent had some graduate work beyond the Master 5 s 
degree, and 1 per cent had the Ph.D. degree. 

iilthough these and other considerations must be taken into account, 
it is here posited that if more adequate criterion data were obtained 
for the scientists participating in the Form C-l study even at this time, 
which is now some two years after the BI 5 s were completed, more satisfactory 
validation results would be obtained. This statement is especially supported 
by several of the findings reported in this study, where it was shown 
that criterion data obtained after oniy one year of observation was 
less reliable and relevant than that obtained in the previous concurrant 
validity studies where the observations were based upon a much longer 
period of time. Nevertheless, it is here concluded that the BI has been 
demonstrated to be a valuable predictive instrument considering the 
difficulty and importance of the predictions made, and could contribute 
valuable information to the identification of scientific talent in a 
carefully designed, selection program. 



9 . 2 . 

Chapter f> 

Examples of Item Content of the Biographical Inventory 
jfn this section a few examples of some of the better biographical items 
will be presented with a brief discussion of the types of items which have 
generally failed to contribute to the identification of scientific talent across 
the studies completed to date* It should be remembered tlst the following 
relationships are characteristic only of the majority? there would be some 
individuals whoso responses to each item would be exceptions to the general finding* 
All of the items cited were related to the Creativity Criterion and on 
occasions to other criteria as well. Since there may be some distortion in the 
responses of the subjects, the extent to which the subjects 5 responses corres- 
pond to the actual situation described remains to be determined. This is an 
important question to which further research should be directed. Results by 
Cline, Richards, Abe, and Needham (1963) indicate that biographical items 
completed by high school students do not describe the home environment in the 
same way as the parents see it. This, of course, does not say whoso perception 
is correct. 

A number of items demonstrated that characteristics of self-determination 
and an individualistic orientation (or inner-directedness) are positively 
related to the criteria. A facet of this is concerned with how the individual 
scientist elects to expend his energies and to what area of his life he devotes 
himself. For example, a definite task-orientation appears to be involved in the 
following question. If an individual responds that, to a great extent, he is the 
kind of person who becomes so absorbed in his work and interests that he does 
not mind a lack of friends, this response was positively related to the criteria, 
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whereas another person® s response that this does not describe him at all w as 
negatively related to the criteria. Another example of an item in this area is 
as follows: :: Assume you are in a situation in which the following two a. ?1 - * 

alternative courses of action arise. Which one of the two would you be most 
likely to do? (A) Bo a good team man so that others like to work with me, 
or (B) Gain a reputation through controversy, if necessary^ as one whose 
scientific word can be trusted.” Response A was correlated negatively with the 
criteria, and Response B was positively related. Wherever this attitude of 
independence originated it evidently tended to have been present during the 
student® s academic career. For example, if the scientist reported that he 
questioned his professors on subject matter considerably more often than 
average, his response was positively related to the criteria. 

The relationship of undergraduate college grade-point average to success 
as a scientist has been shown by many investigations to at best be low? however, 
occasionally a few items in the Academic Section of the Biographical Inventory 
which are concerned with self-reported academic prformance emerge with a low 
but consistent relationship to the Creativity Criterion. For example, a B.A. 
or B. S. degree or less has a negative relationship to the criteria, whereas 
obtaining the Ph.D* degree has a positive relationship. If a scientist 
described his college undergraduate work as being well above average and him- 
self as being satisfied with his progress, this response was positively related 
to the criteria. If a scientist reported that as a student he succeeded exception- 
ally well in his engln eering courses or biological science courses, this had a 
positive relationship to the criteria, while a response of succeeding fairly 
well had a negative relationship. Other items, such as those concerned with 
success in the relevant appearing fields of mathematics , physics , and chemistry, 
have not consistently shown a relationship with the Creativity Criterion. 
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One of the more consistently surprising items which has demonstrated a 
positive relationship to creativity is an item which is concerned with attitudes 
toward making repairs around the house prior to the age of 18, If the subject 
responds that he had a strong dislike for making such repairs, this response 
was positively related to creative performance. It is suspected that this item 
is related to the personality factor of femininity. Previous research has shown 
that this dimension has some relationship to creativity. It may also ref bet 
certain sensitivities and an orientation toward ideas and theoretical approaches 
as opposed to more tangible and mechanical interests. 

This discussion would not be complete without a brief statement of the 
types of items that have failed to discriminate. Generally speaking, items that 
measure a small specific segment of previous experience or a specific fact in 
one’s life history have not been fruitful. For example, items, such as the 
extont of participation in childhood job enterprises such as cutting lawns, 
washing cars, etc., or the number of timos that the subject had changed 
residences by the time hoentered college, or the age at which ne held his first 
paying job, or the highest level of achievement he obtained in the Boy Scouts-, 
have not survived the validation process. .Another area which has so far proved 
barren for identifying scientific talent concerns descriptions of various 
parental characteristics, such as the parents 0 dominance, affection, encouragement, 
strictness, permissiveness, etc. yJhile it is expected that this is an area of 
definite importance, it has proved to be extremely difficult to cultivate 
successfully. One of the reasons for this is probably the complex network of 
interactions that exists between the subject’s parents, so that when any one 
facet of their .’behavior has been measured it does not provide enough 
information about how the other parental characteristics interacts thus, by 
itself the parental characteristic being measured appears unimportant. 
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It would be difficult to estimate the number of items which have either 
been tried out in one form of our inventory or have been carefully examined 
for their potential discriminating power. Certainly the number exceeds 1,000 
items. Undoubtedly it would be possible to construct additional valid items 
to add to the Biographical Inventory, but according to our current understanding 
and measurement skills most of the fertile ground has already been plowed. 
Consequently, gains in the near future through item construction will probably 
be small, although not necessarily unimportant. 
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Chapter 7 

Other Research Activities with Biographical Information 

Identifying Scientific Talent with Biographical I nformation in 
-Other 8 - Settings and Samples . Because of the consistently promising and 
positive results in studies where biographical information was used in 
developing predictive keys (such as predicting creativity among NASA scientists, 
etc. ) there has been an increasing use of psychological instruments of this 
type in a broad variety of settings. One of the direct outcomes of this has 
been the initiation of : r a rosoarcb project, based upon this previous NiiSA 
effort by Ri chard son-Merr ell, Inc., an industrial pharmaceutical company that is 
concerned with the development of a biographical inventoiy for the identification 
of scientific talent, especially in recruiting new personnel. Just completed 
in this study (Cline and Tucker, 1965 ) 9 a biographical inventory (constructed 
by C. tf. Taylor and R. L. Ellison for research use in industry) was administered 
to a large number of scientists and criterion information was obtained having to 
do with creativity and general scientific competence. Each participating 
scientist was rated by his supervisor, his peers, and in some cases his subordinates. 
Results of this study showed that the empirical keys, constructed during this 
study, obtained high initial validities, the correlations ranging in the .70® s 
and .80® s and cross validity predictions of the criteria in the * 30 ®s and .40® s. 

Also, since the biographical information items used in this study contained many 
of the same items that were also used in the NASA studies, it was possible to 
score the test protocols of the pharmaceutical scientists with the NASA 
derived keys.^ The results of this procedure proved definitely significant 

5[t might be mentioned that strict security has been maintained in using 
these keys for this research purpose only. 
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and have far ranging implications and application to this entire area of 
research. It was found that tho NASA Creativity Key (Form A, Total Score Key), 
when applied to the biographical information responses of the Richardson-Merrell 
pharmaceutical scientists, cross validated *35 with the Creativity Rating 
Criterion of these scientists. The NASA Off-Line Key (a correction key) 
cross validated ,10 with the Creativity Criterion, and these two keys had a 
correlation of «,22 with each other. When these two keys were taken together 
in a multiple correlation with the Creativity Criterion, a multiple cross- 
prediction correlation of ,39 resulted. This finding indicates a significant and 
impressive applicability of the results of the NASA studies and the wide 
genoralizeability of the NASA key's across very diverse samples of scientists 
(in this case from the physical to biological sciences and from a government 
agency to private industry). This study therefore suggests a high potential 
value of biographical information in the identification of scientific talent in 
a broad variety of organizational settings. 

Chambers (1964) studied personality and biographical factors of mature 
scientists who are highly creative in research work and those who are much less 
creative. Along wit]?- significant differences between creative scientists and 
their controls on several personality variables, he also found significant 
differences for 16 biographical items , Re was thus able to present a 
biographical and personality profile of those highly creative and not so 
creative scientists whom he studied. 
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W. A. Owens and his associates (Kulberg & Owens, I960; Morrison, Owens, 
Glennon, and Albright, 1952) have been involved in several studies of engineers 
and scientists with biographical data in relation to various aspects of creativity, 
professional interests, and research competence. In a related study, Albright 
and Glennon (1961) found that biographical information could discriminate 
between supervisory and research oriented scientists at all levels of a 
laboratory organization. Also, Smith, Albright, and Glennon (1961) demonstrated 
the value of the personal history technique in the prediction of criteria of 
scientific competence and creativity with a highly select group of research 
scientists. 

In a recent study by McDermid (19&5) with the technical and engineering 
personnel of the Hammond Organ Company, it was found that of the several 
predictive instruments used, only biographical data proved to be significant 
as predictors of both supervisory and peer ratings of creative performance. 

The June 1965 national research conference on the biographical inventory 
approach chaired by E. R, Henry and supported by the Richardson Foundation, 
discovered from practically all 16 participants that across professional 
and other high level, complex fields, the biographical inventory is at least 
as good and is usually better than other devices in predicting job perfor- 
mance. Also, a biographical and multiple criterion study has been tentatively 
planned by McPherson and Cooley with the Dow Chemical Corporation personnel. 

Thus, it appears that the potential value and promise of biographical 
information is now being recognized in many studies across very diverse criterion 
groups. 
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Heter°^en e ity of the Biographical Inventory . Recently a -*niall"pioc6'*of 
exploratory research was completed which highlights the complexity and the 
stability of the Biographical Inventory* In Form C, 12 landmark personality 
items were selected from French (1953) in order to relate the biographical items 
to previous research findings on typical personality instruments. These landmark 
factor items were as follows! Dominance, Interest in Philosophy, Masculinity- 
femininity, Persistence, Self-confidence, Sensitive Attitude, Sociability, 
Gregariousness, Emotionality, Autistic Thought, and Intelligence. In an 
exploratory attempt to see how the biographical items related to these dimensions 
of personality, six of the landmark personality items were held up as criterion 
scores in an item analysis. A 1 though the landmark items themselves had a very j 
low correlation with the regular criterion measures of Creativity, Productivity, 
etc, , scores from the biographical keys which were built to predict these 
personality landmark items correlated substantially higher with the regular 
criterion measures. For example, the item measuring Dominance correlated .12 
with Creativity^ yet the Dominance score from the biographical key of approximately 
items correlated ,37 with the Creativity Criterion. Thus, even though the 
Dominance Criterion item had a very low relationship with the Creativity 
Criterion, the key built to predict the Dominance item also succeeded in predict- 
ing the Creativity Criterion. This was to be expected, at least to some extent, 
because of the much greater length (?4 items versus 1 item) and thus the conse- 
quent greater reliability. Another probable reason is that many of the items which | 
went into the lengthened key were complex factorially and thus measured other J 

aspects of the Creativity Criterion. This is supported by the fact that scores 
from the Dominance Key correlated .83 with the biographical scores that were 



specially keyed for the Creativity Criterion. The cross-validity coefficient 
of this same empirically keyed Dominance score against the Dominance Criterion 
item was .50. Of the other five landmark items which were held up as criteria in 
an item analysis, all were predicted with cross validities ranging from .38 to 
.62, the majority being in the .50°s. These findings again illustrate the 
complexity of the Inventory as it predicted these different areas of personality. 
This procedure of keying biographical items against a landmark personality item 
provides an efficient way of building a longer test for that personality 
characteristic, 

Nielsen (1963) has recently found that a biographical inventory which he 
factored was made up of some twenty to thirty relatively independent dimensions , 
This again is impressive evidenceof the diversity and complexity of the inventory 
and why it is able to predict such a variety of criteria, including complex ones. 

Abe (1963) has recently completed a study which is relevant in this area. 

In Abe«s study one of the correlation matrices from the Form A study discussed 
previously was factor analyzed. This matrix was made up of forty-seven variables 
including seven criteria. The other forty variables were biographical keyed 
scores. Abo found nine factors which ranged from a pure criterion factor to an 
outer-directedness factor which seemed to reflect a desire on the part of 
some scientists for recognition and a need for visibility. This study again 
illustrates the variety of interesting useful information which can be 
obtained through research with biographical information. 
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Biog raphical Information and the I d entification of Scien tific. Talent in 
Y ounger Age Groups « A question has been raised repeatedly that the items in 
the Biographical Inventory aro primarily concerned with topics which are pertinent 
only to adults and to actual on-going research activities* This question implies 
that these biographical items would not bo very appropriate or valid if 
administered to younger age groups, such as high school students or college 
seniors, because the items are oriented too much toward the activities of mature 
research scientists, 

A recent study on this problem supported by the National Science Foundation 
(C. Vf. Taylor, Cooley, and Nielson, 1963) highlights some of the complexities 
involved in early identification of scientific talent, since it implies that 
our present educational program is not geared to give the most appropriate kind of 
training as far as creative scientific achievement is concerned. In the NSF- 
supported summer science program for high school students, some of the students 
have the unusual opportunity to participate full time in research activities. 
Others participate in classroom activities only to learn important science 
materials not in the regular curriculum, Th© main interest of this high school 
study, was to determine whether the creative and productive characteristics 
found for scientists on the job, as discovered in recent studies of Air Force, 
Space Agency, and other scientists, are measurable on high school students in 
these programs and whether these same characteristics are more related to the 
performance cf the students in these research activities than to classroom-only 
performances. 



The data analysis has revealed that two distinct groups can be identified, 
a research achievement group and an academic achievement group* In general 
the predictors with positive validities for the academic program tended to have 
low, zero, or negative validities in the research programs, and vice versa* 

In this study the Biographical Inventory was modified to be appropriate for 
younger age groups. It was found that the vast majority of the items could be used 
without modification 5 a few were revised, and a few wore dropped. Because some 
items had to be rewritten, the scoring keys were constructed on mature 
scientists and predictive (short-range follow-up) validities rather than 
concurrent va li dities were to be determined, it was expected that the revised 
Biographical Inventory would not work very well, if at all, under the circumstances. 
The results indicated that of all the instruments used in this study, the Biogra- 
phical Inventory was the best overall predictor of creative performance. In one 
of the research participation groups in which it was felt that the most valid 
criteria was obtained the Biographical Inventory scores correlated M with 
supervisory ratings on creativity. Needless to say, this i a remarkably satis- 
factory cross validation finding. Certain biographical keys that worked well 
in the research programs did not work well for the academic programs and vice 
versa. The two extreme examples are that the scores from the Professional 
Self-confidence Key were valid for two-thirds of the criteria in the research 
programs but had no significant validities whatsoever in the academic sample 
whereas scores from the Miscellaneous Biographical Key were as good as any 
biographical scores in the academic programs but had no significant validities 



in the research programs. 



It was thought that the Biographical Inventories constructed for 

administration to NASA scientists would probably be more appropriate for 

college seniors than for high school students, since college seniors more closely 

resemble the adult samples upon which the Biographical Inventory was developed* 

Some data have been obtained on this latter issue already, although the criteria 

were not as directly pertinent as those used in the study of high school students* 

Victor Bunderscn has been working on an evaluation of present and potential 

fellowship selection information at the University of Utah, where the Research 

Committee awards approximately 40 graduate fellowships per year* Selection 

has been based largely on grade-point average and open-ended letters of 

recommendation* This study seeks to sraluate these sources of information for 

fellowship selection in the light of various criteria of graduate student perfor- 

in 

mance* As a part of this research, a modified Biographical Inventory was 
administered for research purposes only to a nmmber of seniors and graduate students;: 
who applied for a fellowship. Ratings were obtained a year later on their 
graduate student performances, including their research potential. Again the 
Biographical Inventory scores proved to be the most valid predictor of thes'- | 

multiple criteria? in fact, early indications are that the Biographical 
Inventory, by itself, overshadows the validity of the official, collective 
judgment of the Fellowship Committee who had used the entire folder of materials 
for each applicant in making their fellowship decisions* 

Future Bioffraohical Research Activities* An examination oi the different 
types of items included in the various Biographical Inventories shows that in 
the number of characteristics measured, they are very heterogeneous and complex. 

One of the activities recently completed is an intercorrelation and factor 
analysis of the biographical items, along with appropriate criterion scores* 

Such an analysis will yield a great deal of information about this type of inventor 
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Of special interest is the possibility that factor analysis will contribute to 
the development of more independent and efficient subscores within the in- 
ventory than our existing subjective classification of Developmental History, 
Parents and Family Life, etc., has yielded. This in turn should contribute 
higher validity coefficients derived from combined subscores, thus increasing 
the predictive potential of the Biographical Inventory. Another type of 
useful information from a factor analysis of the items will be the identifi- 
cation of the most promising areas in the inventories. From these leads, 
it should bo possible to construct new items and thereby further improve the 

instrument. 

The Biographical Inventory has usually been found to measure somewhat 
different criterion variance than other traditional types of selection tests. It 
therefore seems advisable to consider research on s:ome of these other kinds 
of measures found to have promise in research on creative scientific talent to 
see how well they supplement the Biographical Inventory scores. Such additional 
validation work could take full advantage of the criterion data as well as the 
biographical data already available on NASA scientists. 

In summary, all our research results obtained to date indicate that 
biographical information is a very promising, if not the most promising, single 
means of identifying creative and other types of scientific talent. 

The cross-validity coefficients obtained are considerably higher than those 
typically reported for the prodiction and identification of creative or of other 
types of scientific talent, which use other kinds of predictors such as high-level 
aptitude tests, intelligence measures, college grade-point averages, and per- 
sonality test measures. It is our conviction that continued research should be 
carried out to exploit thoroughly the potential in the biographical approach so 
the identification of creative scientific talent can be accomplished with as much 



accuracy as possible. 
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